Berthold Heiligers

30 Approximate designs for polynomial regression: Invariance, admissibility, and optimality

Publisher Summary This chapter focuses on approximate linear regression design. It provides an overview of the invariance and admissibility of designs, their interrelations and their implications to design optimality, including numerical algorithms. Invariance structures combined with results on admissibility provide a tool to attack optimal design problems of high dimensions, as occurring for second and third order multiple polynomial models. Invariance with respect to infinite (though compact) transformation and matrix groups usually calls for the Haar probability measures, involving deep measure theoretic results. The Karlin–Studden necessary condition on the support of an admissible design is known to become useless for regression models involving a constant term. Well known numerical algorithms for solving extremum problems in optimal design are pure gradient methods, among them the steepest descent method of Fedorov and Wynn. However, after a quick but rough approximation towards the optimum they become very inefficient. The Quasi–Newton methods of Gaffke and Heiligers provide an efficient way of computing the optimum very accurately.

Minimax designs for estimating the optimum point in a quadratic response surface

Abstract We consider designs when interest is in estimating the optimal factor combination in a multiple quadratic regression setup, supposing that this factor combination belongs to a given set. By involving the concepts of admissibility and invariance of designs we substantially reduce the problem of calculating minimax designs. Exemplary, we give optimal designs for some setups on the ball and on the cube.

Computing optimal approximate invariant designs for cubic regression on multidimensional balls and cubes

Abstract We consider the problem of computing numerically optimal approximate designs for cubic multiple regression in v ≥ 2 variables under a given convex and differentiable optimality criterion, where the experimental region is either a ball or a symmetric cube, both centered at zero. We restrict to designs which are invariant w.r.t. the group of permutations and sign changes acting on the experimental region. For many optimality criteria this is justified by the fact that within the class of all designs there exists an optimal one which is invariant. In particular, this holds true for any convex and orthogonally invariant criterion, including Kiefers фp-criteria, but also for integrated variance criteria (I-criteria) and for mixtures thereof. Based on the mathematical tools developed by Gaffke and Heiligers (Metrika 42 (1995) 29–48), we show how the algorithms of Gaffke and Mathar (Optimization 24 (1992) 91–126) can be applied to compute (nearly) optimal designs. As examples, we present numerical results for (mixtures of) Kiefers фp-criteria under a fixed regression model, which show that the proposed methods work well. For the case that the experimental region is a ball (centered at zero) we also apply the algorithms to the subclass of (third order) rotatable designs, and evaluate the efficiency loss of optimal rotatable designs. Finally, we examine several strategies for obtaining exact designs from optimal approximate ones by changing the weights to integer multiples of 1 n .

Admissible experimental designs in multiple polynomial regression

Abstract We give a connection between admissibility of a design in a multiple polynomial setup in v variables and admissibility of subdesigns in multiple setups in less than v variables. This leads to practicable conditions on the support of an admissible design. Especially, for a convex experimental region X an admissible linear regression design is concentrated on the extreme points of X , and an admissible quadratic regression design has at most one support point in the relative interior of an arbitrary face of X . For the ball and the cube we give all admissible (and invariant) linear and quadratic regression designs, proving that the necessary conditions on the support given in Heiligers (1991) are sufficient as well. As demonstrated for both regions, our results substantially reduce the problem of finding optimal designs.

E-optimal designs for polynomial spline regression

We give the E-optimal approximate designs for mean (sub-) parameters in dth degree totally positive polynomial spline regression with prescribed knots over an arbitrary compact real interval. Based on a duality between E- and scalar optimality, the optimal design is found to be supported by the extrema of the Chebyshev (i.e., equi-oscillating) spline, with corresponding weights given in terms of certain Lagrange interpolation splines. In particular, for dth degree polynomial regression, parameterized w.r.t. a totally positive basis (e.g. the Bernstein polynomials), we obtain the solution to the E-optimal design problem, where, contrary to the ordinary monomial setup, no restrictions on the size and location of the regression interval or on the particular system of parameters of interest are required.

Admissibility of experimental designs in linear regression with constant term

Abstract Karlin and Studden (1966a) proved that each support point of an admissible approximate experimental design in a linear regression setup maximizes a nonnegative quadratic form in the vector of regression functions. As noted by them, this result becomes trivial whenever the regression function contains a constant term. The main purpose of the present paper is to establish nontrivial necessary conditions for admissibility in these situations. By a characterization of comparability of two information matrices at a time it is proved that the information matrix of an admissible design maximizes a linear function under certain linear restrictions. This implies that it solves an unconstrained nonconstant linear optimization problem, too, yielding nontrivial conditions on the support points similar to those of the Karlin and Studden result. Exemplary, we consider multiple polynomial regression. For the linear setup we give all ‘invariant’ and admissible designs. In the quadratic case (under the additional assumption of convexity of the experimental region K ) the support points of an ‘invariant’ and admissible design are found to be either zero or special boundary points of K .

Invariant admissible and optimal designs in cubic regression on the v-ball

Abstract When considering problems of design optimality in multiple polynomial regression setups, it may be for computational reasons that attention is restricted to rotatable designs. However, not even for an orthogonal equivariant criterion, a (globally) optimal design needs to be rotatable. In the linear and quadratic cases the results in Heiligers (1992) can be used to solve the unrestricted design problem. In the present paper we will consider the cubic setup on the ball. By combining the concept of admissibility of designs with a less restrictive invariance property (namely the invariance of information matrices w.r.t. permutations and sign changes acting on the experimental region), we will find an essentially complete class of invariant designs. Roughly spoken, for a wide class of criteria (including orthogonal equivariant criteria) it suffices to consider designs which assign mass to at most 2 v special sets of equally weighted points located on the boundary and on (at most) v different spheres in the interior of the ball. Consequently, for these criteria we have to solve a minimization problem in 3 v — 1 variables, where the objective function is convex at least in the 2 v − 1 weights. Exemplary, for some of Kiefers ϕ p -criteria we will give optimal values, and we will compare these with the ϕ p -optimal rotatable designs computed in Galil and Kiefer (1979).

Isotropic discrete orientation distributions on the 3D special orthogonal group

In modeling the linear elastic behavior of a polycrystalline material on the microscopic level, a special problem is to determine a so-called discrete orientation distributions (DODs) which satisfy the isotropy condition. A DOD is a probability measure with finite support on SO(3), the special orthogonal group in three dimensions. Isotropy of a DOD can be viewed as an invariance property of a certain moment matrix of the DOD. So the problem of finding isotropic DODs resembles that of finding weakly invariant linear regression designs. In fact, methods from matrix and group theory which have been successfully applied in linear regression design can also be utilized here to construct various isotropic DODs. Of particular interest are isotropic DODs with small support. Crystal classes with additional symmetry properties are modeled by stiffness tensors having a non-trivial symmetry group. There are six possible non-trivial symmetry groups, up to conjugation. In either cases we find isotropic DODs with fairly small support, in particular for the cubic and the transversal symmetry groups.