Alan Julian Izenman

Recent developments in nonparametric density estimation

Advances in computation and the fast and cheap computational facilities now available to statisticians have had a significant impact upon statistical research, and especially the development of nonparametric data analysis procedures. In particular, theoretical and applied research on nonparametric density estimation has had a noticeable influence on related topics, such as nonparametric regression, nonparametric discrimination, and nonparametric pattern recognition. This article reviews recent developments in nonparametric density estimation and includes topics that have been omitted from review articles and books on the subject. The early density estimation methods, such as the histogram, kernel estimators, and orthogonal series estimators are still very popular, and recent research on them is described. Different types of restricted maximum likelihood density estimators, including order-restricted estimators, maximum penalized likelihood estimators, and sieve estimators, are discussed, where restrictions are imposed upon the class of densities or on the form of the likelihood function. Nonparametric density estimators that are data-adaptive and lead to locally smoothed estimators are also discussed; these include variable partition histograms, estimators based on statistically equivalent blocks, nearest-neighbor estimators, variable kernel estimators, and adaptive kernel estimators. For the multivariate case, extensions of methods of univariate density estimation are usually straightforward but can be computationally expensive. A method of multivariate density estimation that did not spring from a univariate generalization is described, namely, projection pursuit density estimation, in which both dimensionality reduction and density estimation can be pursued at the same time. Finally, some areas of related research are mentioned, such as nonparametric estimation of functionals of a density, robust parametric estimation, semiparametric models, and density estimation for censored and incomplete data, directional and spherical data, and density estimation for dependent sequences of observations.

Modern Multivariate Statistical Techniques

CHAPTER 3 Page 46, line –15: (K × J)-matrix. Page 47, Equation (3.5): −EF should be −EF . Page 49, line –6: R should be <. Page 53, line –7: “see Exercise 3.4” is not relevant here. Page 53, Equation (3.43): Last term on rhs should be ∂yJ ∂xK . Page 60, Equation (3.98): σ should be σ. Page 61, line 8: (3.106) should be (3.105). Pages 61, 62, Equations (3.109), (3.110), and (3.111): The identity matrices have different dimensions — In the top row of each matrix, the identity matrix has dimension r and in the bottom row it has dimension s. Page 62, line 1: “r-vector” should be “(r + s)-vector.” Page 62, Equation (3.111): ΣXY should be ΣY X . Page 64, Equation (3.127): |Σ| should be |Σ|. Page 62, Equation (3.133): I(2.2) should be I(2,2). Page 65, line 8 (2nd line of property 2): Wr should be Wp. Page 65, property 4: Restate as follows. Let X = (X1, · · · ,Xn) , where Xi ∼ Nr(0,Σ), i = 1, 2, . . . , n, are independently and identically distributed (iid). Let A be a symmetric (n×n)-matrix with ν = rank(A), and let a be a fixed r-vector. Let y = Xa. Then, X AX ∼ Wr(ν,Σ) iff yAy ∼ σ aχ 2 ν , where σ 2 a = a Σa. Page 66, Equation (3.143): last term on rhs, +n should be −n2 . Page 67, line 3: Should read tr(TT ) = ∑r i=1 t 2 ii + ∑ i>j t 2 ij . Page 67, line –6: Should read “idempotent with rank n − 1.” Page 67, line –3: bX should be X b. Page 67, Equation (3.148): n should be n − 1.

Reduced-rank regression for the multivariate linear model

The problem of estimating the regression coefficient matrix having known (reduced) rank for the multivariate linear model when both sets of variates are jointly stochastic is discussed. We show that this problem is related to the problem of deciding how many principal components or pairs of canonical variates to use in any practical situation. Under the assumption of joint normality of the two sets of variates, we give the asymptotic (large-sample) distributions of the various estimated reduced-rank regression coefficient matrices that are of interest. Approximate confidence bounds on the elements of these matrices are then suggested using either the appropriate asymptotic expressions or the jackknife technique.

Review Papers: Recent Developments in Nonparametric Density Estimation

Abstract Advances in computation and the fast and cheap computational facilities now available to statisticians have had a significant impact upon statistical research, and especially the development of nonparametric data analysis procedures. In particular, theoretical and applied research on nonparametric density estimation has had a noticeable influence on related topics, such as nonparametric regression, nonparametric discrimination, and nonparametric pattern recognition. This article reviews recent developments in nonparametric density estimation and includes topics that have been omitted from review articles and books on the subject. The early density estimation methods, such as the histogram, kernel estimators, and orthogonal series estimators are still very popular, and recent research on them is described. Different types of restricted maximum likelihood density estimators, including order-restricted estimators, maximum penalized likelihood estimators, and sieve estimators, are discussed, where re...

Philatelic Mixtures and Multimodal Densities

Abstract A century ago, postage stamps were printed on a deliberate mixture of different paper types, each having its own thickness characteristics due to poor quality control in paper manufacture. It was not unusual for different types of paper to be used for a single stamp issue, because ordinary white wove paper was often not readily available in quantities necessary for production of a new stamp issue. Unfortunately, the different paper types used were not always well documented. Each paper type was purchased by the ream, which was required to consist of a specific number of sheets and had to weigh a certain amount. To cut costs, manufacturers often satisfied both requirements by mixing a few “thick” sheets of paper into an otherwise underweight ream. Although the importance of paper thickness in the philatelic literature is reflected by a higher market value for stamps printed on the thicker and more scarce paper, stamp catalogs have been notoriously vague about characterizing paper thickness, relyin...

Linear Discriminant Analysis

Suppose we are given a learning set \(\mathcal{L}\) of multivariate observations (i.e., input values \(\mathfrak{R}^r\)), and suppose each observation is known to have come from one of K predefined classes having similar characteristics. These classes may be identified, for example, as species of plants, levels of credit worthiness of customers, presence or absence of a specific medical condition, different types of tumors, views on Internet censorship, or whether an e-mail message is spam or non-spam.

The Effect of Neighborhood Characteristics and Spatial Spillover on Urban Juvenile Delinquency and Recidivism

The objective of this research is to investigate the relationship between neighborhood characteristics and juvenile delinquency and recidivism (the proportion of delinquents who commit crimes following completion of a court-ordered program) in Philadelphia, Pennsylvania. We acquired data on collective efficacy, socioeconomic character, and crime for input into multivariate ordinary least squares (OLS) and spatial econometric regression analyses. Both delinquency and recidivism are concentrated in impoverished neighborhoods with violent crime, although this relationship is far stronger for delinquency than for recidivism. After accounting for the influence of crime and poverty, OLS regression results suggest that African American neighborhoods tend to exhibit higher delinquency rates, but lower recidivism rates, than other neighborhoods. Spatial lag models of recidivism rate indicate the presence of spatial spillover effects, which renders the influence of neighborhood racial character on recidivism rate not significant and which we speculate represents interaction among juveniles across neighborhood boundaries.

Babies and the blackout: The genesis of a misconception

Abstract Nine months after the great New York City blackout in November 1965, a series of articles in the New York Times alleged a sharp increase in the citys birthrate. A number of medical and demographic articles then appeared making contradictory (and sometimes erroneous) statements concerning the blackout effect. None of these analyses are fully satisfactory from the statistical standpoint, omitting such factors as weekday-weekend effects, seasonal trends, and a gradual decline in the citys birthrate. Using daily birth statistics for New York City over the 6-year period 1961–1966, techniques of data analysis and time-series analysis are employed in this paper to investigate the above effects.

Information criteria and harmonic models in time series analysis

In this paper we present a novel methodology for estimating harmonic models in time series. The amplitude density function is derived from an inversion of the spectral representation of the series and is found to have the property of strong consistency. It is used in conjunction with non-linear least squares regression in an iterative procedure to yield precise estimates of the frequencies in the model. Information criteria are then used in determining how many frequencies to include in the model. The optimal procedure was tested using Monte Carlo techniques, and proved successful at correctly estimating the underlying harmonic model. The methodology was applied to a series of magnitudes of a variable star, to a series of signed sunspot numbers, and to a series of temperature data. A class of iterative procedures was investigated, and an optimal procedure proposed, which involves the use of non-linear least squares and an update of the amplitude density function after each new frequency has been estimated...

Kernel estimation of the survival function and hazard rate under weak dependence

Abstract Let {X(k)} be a strictly stationary stochastic process, where X(k) takes values in Rd and has a probability density f. Let F (x)= 1 -F(x) be the survival function and r(x) be the hazard rate defined by r(x)=f(x)/ F (x) , where F is the distribution function of X(k). Let X1,…,Xn ben consecutive observations of {X(k)} and F n (x) =n − 1 ∑ n j= 1 Y j (x) , where Y j >x= 1 if X j >x , and Yj(x)=0 otherwise, j = 1,…,n. As estimators of F (x) and r(x), we consider F n (x) and ⨍ n (x)/ F n (x) respectively, where ⨍ n (x) is a kernel type estimator of ⨍(x) . Appropriate conditions for the uniform consistency of the estimators are given and sharp rates of convergence are also obtained. The stochastic process is assumed to satisfy certain strong mixing and absolute regularity conditions.