Hubert Schwetlick

Numerical Methods for Estimating Parameters in Nonlinear Models With Errors in the Variables

A numerically stable implementation of the Gauss-Newton method for computing least squares estimates of parameters and variables in explicit nonlinear models with errors in the variables is proposed. The algorithm uses only orthogonal transformations and exploits the special structure of the problem. Moreover, a partially regularized Marquardt-like version is described that works with a reasonable overhead of arithmetic operations and storage compared to the error-free case.

Computing turning points of curves implicitly defined by nonlinear equations depending on a parameter

Let the space curveL be defined implicitly by the (n, n+1) nonlinear systemH(u)=0. A new direct Newton-like method for computing turning points ofL is described that requires per step only the evaluation of one Jacobian and 5 function values ofH. Moreover, a linear system of dimensionn+1 with 4 different right hand sides has to be solved per step. Under suitable conditions the method is shown to converge locally withQ-order two if a certain discretization stepsize is appropriately chosen. Two numerical examples confirm the theoretical results.ZusammenfassungDie RaumkurveL werde implizit durch das nichtlineare (n, n+1)-SystemH(u)=0 definiert. Es wird ein neues direktes Newton-ähnliches Verfahren zur Bestimmung der Rückkehrpunkte vonL beschrieben, das pro Schritt lediglich die Berechnung einer Jacobimatrix und 5 Funktionswerten vonH erfordert. Außerdem ist pro Schritt ein lineares Gleichungssystem der Dimensionn+1 mit 4 verschiedenen rechten Seiten zu lösen. Unter passenden voraussetzungen wird die lokale undQ-quadratische Konvergenz des Verfahrens bewiesen, sofern eine gewisse Diskretisierungsschrittweise geeignet gewählt wird. Zwei numerische Beispiele bestätigen die theoretischen Resultate.

Least squares approximation by splines with free knots

Suppose we are given noisy data which are considered to be perturbed values of a smooth, univariate function. In order to approximate these data in the least squares sense, a linear combination of B-splines is used where the tradeoff between smoothness and closeness of the fit is controlled by a smoothing term which regularizes the least squares problem and guarantees unique solvability independent of the position of knots. Moreover, a subset of the knot sequence which defines the B-splines, the so-calledfree knots, is included in the optimization process.The resulting constrained least squares problem which is linear in the spline coefficients but nonlinear in the free knots is reduced to a problem that has only the free knots as variables. The reduced problem is solved by a generalized Gauss-Newton method. The method developed can be combined with a knot removal strategy in order to obtain an approximating spline with as few parameters as possible.

Iteratively Reweighted Least Squares

In solving robust linear regression problems, the parameter vector x, as well as an additional parameter s that scales the residuals, must be estimated simultaneously. A widely used method for doing so consists of first improving the scale parameter s for fixed x, and then improving x for fixed s by using a quadratic approximation to the objective function g. Since improving x is the expensive part of such algorithms, it makes sense to define the new scale s as a minimizes of g for fixed x. A strong global convergence analysis of this conceptual algorithm is given for a class of convex criterion functions and the so-called H- or W-approximations to g. Moreover, some appropriate finite and iterative subalgorithms for minimizing g with respect to s are discussed. Furthermore, the possibility of transforming the robust regression problem into a nonlinear least-squares problem is discussed. All algorithms described here were tested with a set of test problems, and the computational efficiency was compared wit...

Zur Lösung parameterabhängiger nichtlinearer Gleichungen mit singulären Jacobi-Matrizen

SummaryFor solving the nonlinear systemG(x, t)=0,G|ℝn × ℝ1→ℝn, which is assumed to have a smooth curve ℭ of solutions a continuation method with self-choosing stepsize is proposed. It is based on a PC-principle using an Euler-Cauchy-predictor and Newtons iteration as corrector. Under the assumption thatG is sufficiently smooth and the total derivative (∂1G(x, t)⋮∂2G(x, t)) has full rankn along ℭ the method is proven to terminate with a solution (xN, 1) of the system fort=1. It works succesfully, too, if the Jacobians ∂1G(x, t) become singular at some points of ℭ, e.g., if ℭ has turning points. The method is especially able to give a point-wise approximation of the curve implicitly defined as solution of the system mentioned above.

Ableitungsfreie Verfahren mit höherer Konvergenzgeschwindigkeit

ZusammenfassungFür Gleichungen in mehrdimensionalen Räumen werden Klassen von ableitungsfreien Verfahren angegeben, welche in der Durchführung allein Steigungen erster Ordnung benötigen. Ein Iterationsschritt besteht ausk Stufen, wobei die einzelnen Stufen die Anwendung der vereinfachten Regula falsi bedeuten. Es ergibt sich, daß sich das Maximum des Wirkungsgrades mit wachsender Dimension zu den größerenk-Werten hin verschiebt. Die Konvergenz der Verfahren wird für einfache Nullstellen und hinreichend gute Startwerte nachgewiesen.SummaryFor equations in spaces of several dimensions classes of methods not involving derivatives are given which require only first-order divided differences. One interation step consists ofk stages each of them meaning the application of the modified regula falsi. The maximum of the efficiency index is shown to shift to greater values ofk with increasing dimension. The convergence of the methods is proved for simple zeros and sufficiently good initial values.

Spline smoothing under constraints on derivatives

An efficient algorithm for computing a smoothing polynomial splines under inequality constraints on derivatives is introduced where both order and breakpoints ofs can be prescribed arbitrarily. By using the B-spline representation ofs, the original semi-infinite constraints are replaced by stronger finite ones, leading to a least squares problem with linear inequality constraints. Then these constraints are transformed into simple box constraints by an appropriate substitution of variables so that efficient standard techniques for solving such problems can be applied. Moreover, the smoothing term commonly used is replaced by a cheaply computable approximation. All matrix transformations are realized by numerically stable Givens rotations, and the band structure of the problem is exploited as far as possible.

A modified block Newton iteration for approximating an invariant subspace of a symmetric matrix

In this paper we propose a Modified Block Newton Method (MBNM) for approximating an invariant subspace J and the corresponding eigenvalues of a symmetric matrix A. The method generates a sequence of matrices Z(k) which span subspaces Jk approximating J. The matrices Z(k) are calculated via a Newton step applied to a special formulation of the block eigenvalue problem for the matrix A, followed by a Rayleigh-Ritz step which also yields the corresponding eigenvalue approximations. We show that for sufficiently good initial approximations the subspaces Jk converge to J in the sense that sinϕk with ϕk := ∢(Jk,J)Q-quadratically converges to zero under appropriate conditions

Constrained approximation by splines with free knots

In this paper, a method that combines shape preservation and least squares approximation by splines with free knots is developed. Besides the coefficients of the spline a subset of the knot sequence, the so-calledfree knots, is included in the optimization process resulting in a nonlinear least squares problem in both the coefficients and the knots. The original problem, a special case of aconstrained semi-linear least squares problem, is reduced to a problem that has only the knots of the spline as variables. The reduced problem is solved by a generalized Gauss-Newton method. Special emphasise is given to the efficient computation of the residual function and its Jacobian.

Ein lokal überlinear konvergentes Verfahren zur Bestimmung von Rückkehrpunkten implizit definierter Raumkurven

SummaryA numerical technique is described for computing turning points of a space curveL implicitly defined by a nonlinear system ofn equations inn+1 variables. The basic idea is a local parametrization ofL where the parameter that gives the next iterate is determined by applying one step of the well-known method for minimizing a real function using cubic Hermite interpolation with two nodes. The method is shown to convergeQ-super-linearly and withR-order of at least two. A numerical example concerning the analysis of nonlinear resistive circuits shows the algorithm to work effectively on real life problems.