Wilfried Seidel

A Cautionary Note on Likelihood Ratio Tests in Mixture Models

We show that iterative methods for maximizing the likelihood in a mixture of exponentials model depend strongly on their particular implementation. Different starting strategies and stopping rules yield completely different estimators of the parameters. This is demonstrated for the likelihood ratio test of homogeneity against two-component exponential mixtures, when the test statistic is calculated by the EM algorithm.

Editorial: recent developments in mixture models

Recent developments in the area of mixture models are introduced, reviewed and discussed. The paper introduces this special issue on mixture models, which touches upon a diversity of developments which were the topic of a recent conference on mixture models, taken place in Hamburg, July 2001. These developments include issues in nonparametric maximum likelihood theory, the number of components problem, the non-standard distribution of the likelihood ratio for mixture models, computational issues connected with the EM algorithm, several special mixture models and application studies.

Editorial: Advances in Mixture Models

The importance of mixture distributions is not only remarked by a number of recent books on mixtures including Lindsay (1995), Böhning (2000), McLachlan and Peel (2000) and Frühwirth-Schnatter (2006) which update previous books by Everitt and Hand (1981), Titterington et al. (1985) and McLachlan and Basford (1988). Also, a diversity of publications on mixtures appeared in this journal since 2003 (which we take here as a milestone with the appearance of the first special issue on mixtures) including Hazan et al. (2003), Benton and Krishnamoorthy (2003), Woodward and Sain (2003), Besbeas and Morgan (2004), Jamshidian (2004), Hürlimann (2004), Bohacek and Rozovskii (2004), Tao et al. (2004), Vaz de Melo Mendes and Lopes (2006), Agresti et al. (2006), Bartolucci and Scaccia (2006), D’Elia and Piccolo (2005), Neerchal and Morel (2005), Klar and Meintanis (2005), Bocci et al. (2006), Hu and Sung (2006), Seidel et al. (2006), Nadarajah (2006), Almhana et al. (2006), Congdon (2006), Priebe et al. (2006), and Li and Zha (2006). In the following we give a brief introduction to the papers contributing novels aspects in this Special Issue. These come from a diversity of areas as different as capture–recapture modelling, likelihood based cluster analysis, semiparametric mixture modelling in microarray data, latent class analysis or integer lifetime data analysis—just to mention a few. Mixture models are frequently used in capture–recapture studies for estimating population size (Chao, 1987; Link, 2003; Böhning and Schön, 2005; Böhning et al., 2005; Böhning and Kuhnert, 2006). In this issue, Mao (2007) highlights a variety of sources of difficulties in statistical inference using mixture models and uses a binomial mixture model as an illustration. Random intercept models for binary data—as useful tools for addressing between-subject heterogeneity—are discussed by Caffo et al. (2007). The nonlinearity of link functions for binary data is blurred in probit models with a normally distributed random intercept because the resulting model implies a probit marginal link as well. Caffo et al. (2007) explore another family of random intercept models where the distribution associated with the marginal and conditional link function as well as the random effect distribution are all of the same family. Formann (2007) extends the latent class approach (as a specific discrete multivariate mixture model) for situations where the discrete outcome variables (such as longitudinal binary data) experience nonignorable associations and, in addition and most importantly, have missing entries as it is rather typical for repeated observations in longitudinal studies. The modelling also incorporates potential covariates. This is illustrated using data from the Muscatine Coronary Risk Factor Study. The contribution by Grün and Leisch (2007) introduces the R-package flexmixwhich provides flexible modelling of finite mixtures of regression models using the EM algorithm. Alfò et al. (2007) consider a semiparametric mixture model for detecting differentially expressed genes in microarray experiments.An important goal of microarray studies is the detection of genes that show significant changes in observed expressions when two or more classes of biological samples (e.g. treatment and control) are compared. With the c-fold rule a gene is declared to be differentially expressed if its average expression level varies by more than a constant (typically 2). Instead, Alfò et al. (2007) introduce a gene-specific random term to control for both dependence among genes and variability with respect to the probability of yielding a fold change over a threshold c. Likelihood based inference is accomplished with a two-level finite mixture model while nonparametric Bayesian estimation is performed through the counting distribution of exceedances. Mixtures-of-experts models (Jacobs et al., 1991) and their generalization, hierarchical mixtures-of-expert models (Jordan and Jacobs, 1994) have been introduced to account for nonlinearities and other complexities in the data.

EditorialAdvances in Mixture Models

The importance of mixture distributions is not only remarked by a number of recent books on mixtures including Lindsay (1995), Böhning (2000), McLachlan and Peel (2000) and Frühwirth-Schnatter (2006) which update previous books by Everitt and Hand (1981), Titterington et al. (1985) and McLachlan and Basford (1988). Also, a diversity of publications on mixtures appeared in this journal since 2003 (which we take here as a milestone with the appearance of the first special issue on mixtures) including Hazan et al. (2003), Benton and Krishnamoorthy (2003), Woodward and Sain (2003), Besbeas and Morgan (2004), Jamshidian (2004), Hürlimann (2004), Bohacek and Rozovskii (2004), Tao et al. (2004), Vaz de Melo Mendes and Lopes (2006), Agresti et al. (2006), Bartolucci and Scaccia (2006), D’Elia and Piccolo (2005), Neerchal and Morel (2005), Klar and Meintanis (2005), Bocci et al. (2006), Hu and Sung (2006), Seidel et al. (2006), Nadarajah (2006), Almhana et al. (2006), Congdon (2006), Priebe et al. (2006), and Li and Zha (2006). In the following we give a brief introduction to the papers contributing novels aspects in this Special Issue. These come from a diversity of areas as different as capture–recapture modelling, likelihood based cluster analysis, semiparametric mixture modelling in microarray data, latent class analysis or integer lifetime data analysis—just to mention a few. Mixture models are frequently used in capture–recapture studies for estimating population size (Chao, 1987; Link, 2003; Böhning and Schön, 2005; Böhning et al., 2005; Böhning and Kuhnert, 2006). In this issue, Mao (2007) highlights a variety of sources of difficulties in statistical inference using mixture models and uses a binomial mixture model as an illustration. Random intercept models for binary data—as useful tools for addressing between-subject heterogeneity—are discussed by Caffo et al. (2007). The nonlinearity of link functions for binary data is blurred in probit models with a normally distributed random intercept because the resulting model implies a probit marginal link as well. Caffo et al. (2007) explore another family of random intercept models where the distribution associated with the marginal and conditional link function as well as the random effect distribution are all of the same family. Formann (2007) extends the latent class approach (as a specific discrete multivariate mixture model) for situations where the discrete outcome variables (such as longitudinal binary data) experience nonignorable associations and, in addition and most importantly, have missing entries as it is rather typical for repeated observations in longitudinal studies. The modelling also incorporates potential covariates. This is illustrated using data from the Muscatine Coronary Risk Factor Study. The contribution by Grün and Leisch (2007) introduces the R-package flexmixwhich provides flexible modelling of finite mixtures of regression models using the EM algorithm. Alfò et al. (2007) consider a semiparametric mixture model for detecting differentially expressed genes in microarray experiments.An important goal of microarray studies is the detection of genes that show significant changes in observed expressions when two or more classes of biological samples (e.g. treatment and control) are compared. With the c-fold rule a gene is declared to be differentially expressed if its average expression level varies by more than a constant (typically 2). Instead, Alfò et al. (2007) introduce a gene-specific random term to control for both dependence among genes and variability with respect to the probability of yielding a fold change over a threshold c. Likelihood based inference is accomplished with a two-level finite mixture model while nonparametric Bayesian estimation is performed through the counting distribution of exceedances. Mixtures-of-experts models (Jacobs et al., 1991) and their generalization, hierarchical mixtures-of-expert models (Jordan and Jacobs, 1994) have been introduced to account for nonlinearities and other complexities in the data.

Likelihood ratio tests based on subglobal optimization: A power comparison in exponential mixture models

The paper compares several versions of the likelihood ratio test for exponential homogeneity against mixtures of two exponentials. They are based on different implementations of the likelihood maximization algorithm. We show that global maximization of the likelihood is not appropriate to obtain a good power of the LR test. A simple starting strategy for the EM algorithm, which under the null hypothesis often fails to find the global maximum, results in a rather powerful test. On the other hand, a multiple starting strategy that comes close to global maximization under both the null and the alternative hypotheses leads to inferior power.

Theory & Methods: Testing for Homogeneity in an Exponential Mixture Model

This paper studies diagnostic procedures to test for homogeneity against unobserved heterogeneity in an exponential mixture model. The procedures include a dispersion score test, a likelihood ratio test, a moment likelihood approach and several goodness-of-fit tests. The paper compares the empirical power of these tests on a broad range of alternatives and proposes a new test that combines the dispersion score test with a properly chosen goodness-of-fit procedure; its empirical power comes close to the power of the best of the other tests.

Types of likelihood maxima in mixture models and their implication on the performance of tests

In two-component mixtures of exponential distributions, different strategies for starting the likelihood maximization algorithm converge to different types of maxima. The power of an LR test of homogeneity against such a mixture strongly depends on the considered strategy, and global maximization need not result in the largest power. An explanation is given on basis of a systematic investigation of the likelihood function in a large number of simulations, using a variety of diagnostic tools. Thereby, we also gain a deeper insight into the properties of the samples that generate particular types of solutions of the likelihood equation. In particular, “spurious solutions” often occur; these are mainly responsible for the fact that global maximization may not result in a statistically meaningful estimator. Removing the smallest elements of a sample may drastically increase the power of previously inferior strategies.

Efficient calculation of the NPMLE of a mixing distribution for mixtures of exponentials

A crucial step in all gradient-based algorithms for calculating the nonparametric maximum likelihood estimator in mixture models is the global maximization of the gradient function. For example, in mixtures of exponentials, the methods usually proposed fail. Based on a discretization which is adapted to the data points, a method for maximizing the gradient is suggested. The method is implemented in different gradient-based algorithms; a comparison shows that on mixtures of exponentials, the ISDM algorithm introduced by Lesperance and Kalbfleisch is much faster than its competitors.

A possible way out of the pitfall of acceptance sampling by variables: treating variances as unknown

Abstract According to v. Collanis critique, the classical model of acceptance sampling by variables for individual lots is wrong. On the other hand, it may be considered as an approximation for large lots. Investigating the approximation error for small lots, it turns out that models with unknown variance perform better than those with known variance. For very small lots, it is often better to use a test based on unknown variance, even if the latter is known. The reason is that by estimating the variance, one obtains under certain conditions a test statistic that follows more closely the fraction nonconforming in the lot.

Testing Against Nonparametric Alternatives in Mixture Models

Likelihood ratio tests in parametric mixture models suffer from several sources of instability, therefore tests against a nonparametric alternative are proposed. Their performance depends on the strategies for likelihood maximization. This article develops a fast and statistically powerful combination of methods under the null hypothesis and under the alternative hypothesis; the method also includes elimination of spurious components. Modifying a strategy proposed by McLachlan, a sequence of such tests is applied for assessing the number of components and its performance is analyzed in a number of simulation studies in exponential mixture models. Although critical values have to be bootstrapped, the probability of overestimating is still bounded by the nominal level of the individual tests. Taking into account the number of components that can be reliably detected on the basis of a certain sample size, the proposed procedure yields the minimum number of components that is required for an adequate representation of the sample. It is compared to methods based on the Bayesian and the Akaike information criterion, and a recommendation is given to identify a range of meaningful model orders compatible with a given sample.