Felipe Cucker

On the mathematical foundations of learning

(1) A main theme of this report is the relationship of approximation to learning and the primary role of sampling (inductive inference). We try to emphasize relations of the theory of learning to the mainstream of mathematics. In particular, there are large roles for probability theory, for algorithms such as least squares, and for tools and ideas from linear algebra and linear analysis. An advantage of doing this is that communication is facilitated and the power of core mathematics is more easily brought to bear. We illustrate what we mean by learning theory by giving some instances. (a) The understanding of language acquisition by children or the emergence of languages in early human cultures. (b) In Manufacturing Engineering, the design of a new wave of machines is anticipated which uses sensors to sample properties of objects before, during, and after treatment. The information gathered from these samples is to be analyzed by the machine to decide how to better deal with new input objects (see [43]). (c) Pattern recognition of objects ranging from handwritten letters of the alphabet to pictures of animals, to the human voice. Understanding the laws of learning plays a large role in disciplines such as (Cognitive) Psychology, Animal Behavior, Economic Decision Making, all branches of Engineering, Computer Science, and especially the study of human thought processes (how the brain works). Mathematics has already played a big role towards the goal of giving a universal foundation of studies in these disciplines. We mention as examples the theory of Neural Networks going back to McCulloch and Pitts [25] and Minsky and Papert [27], the PAC learning of Valiant [40], Statistical Learning Theory as developed by Vapnik [42], and the use of reproducing kernels as in [17] among many other mathematical developments. We are heavily indebted to these developments. Recent discussions with a number of mathematicians have also been helpful. In

Emergent Behavior in Flocks

We provide a model (for both continuous and discrete time) describing the evolution of a flock. Our model is parameterized by a constant beta capturing the rate of decay-which in our model is polynomial-of the influence between birds in the flock as they separate in space. Our main result shows that when beta<1/2 convergence of the flock to a common velocity is guaranteed, while for betages1/2 convergence is guaranteed under some condition on the initial positions and velocities of the birds only

Best Choices for Regularization Parameters in Learning Theory: On the Bias—Variance Problem

Abstract. No abstract.

Avoiding Collisions in Flocks

Among the many models for flocking systems of interacting particles, the one introduced by Cucker and Smale has attracted attention due to the fact that a convergence to flocking (i.e., to a common velocity) could be established depending on conditions on the initial state of the system. In this note we extend this model by adding to it a repelling force between particles. We show that, for this modified model, convergence to flocking is established along the same lines while, in addition, avoidance of collisions (i.e., the respect of a minimal distance between particles) is ensured.

On mixed and componentwise condition numbers for Moore–Penrose inverse and linear least squares problems

Classical condition numbers are normwise: they measure the size of both input perturbations and output errors using some norms. To take into account the relative of each data component, and, in particular, a possible data sparseness, componentwise condition numbers have been increasingly considered. These are mostly of two kinds: mixed and componentwise. In this paper, we give explicit expressions, computable from the data, for the mixed and componentwise condition numbers for the computation of the Moore-Penrose inverse as well as for the computation of solutions and residues of linear least squares problems. In both cases the data matrices have full column (row) rank.

Modeling Language Evolution

Abstract We describe a model for the evolution of the languages used by the agents of a society. Our main result proves convergence of these languages to a common one under certain conditions. A few special cases are elaborated in more depth.

A new condition number for linear programming

Abstract.In this paper we define a new condition number ?(A) for the following problem: given a m by n matrix A, find x∈ℝn, s.t. Ax<0. We characterize this condition number in terms of distance to ill-posedness and we compare it with existing condition numbers for the same problem.

Complexity estimates depending on condition and round-off error

This paper has two agendas. One is to develop the foundations of round-off in computation. The other is to describe an algorithm for deciding feasibility for polynomial systems of equations and inequalities together with its complexity analysis and its round-off properties. Each role reinforces the other.

Counting complexity classes for numeric computations II: Algebraic and semialgebraic sets

We define counting classes #PR and #PC in the Blum-Shub-Smale setting of computations over the real or complex numbers, respectively. The problems of counting the number of solutions of systems of polynomial inequalities over R, or of systems of polynomial equalities over C, respectively, turn out to be natural complete problems in these classes. We investigate to what extent the new counting classes capture the complexity of computing basic topological invariants of semialgebraic sets (over R) and algebraic sets (over C). We prove that the problem of computing the Euler-Yao characteristic of semialgebraic sets is FPR#PR-complete, and that the problem of computing the geometric degree of complex algebraic sets is FPC#PC-complete. We also define new counting complexity classes in the classical Turing model via taking Boolean parts of the classes above, and show that the problems to compute the Euler characteristic and the geometric degree of (semi)algebraic sets given by integer polynomials are complete in these classes. We complement the results in the Turing model by proving, for all k ∈ N, the FPSPACE-hardness of the problem of computing the kth Betti number of the set of real zeros of a given integer polynomial. This holds with respect to the singular homology as well as for the Borel-Moore homology.

Computing over the Reals with Addition and Order

This paper deals with issues of structural complexity in a linear version of the Blum-Shub-Smale model of computation over the real numbers. Real versions of PSPACE and of the polynomial time hierarchy are defined, and their properties are investigated. Mainly two types of results are presented: ?Equivalence between quantification over the real numbers and over {0, 1};?Characterizations of recognizable subsets of {0, 1}* in terms of familiar discrete complexity classes. The complexity of the decision and quantifier elimination problems in the theory of the reals with addition and order is also studied.