K. Drosou

A New Variable Selection Approach Inspired by Supersaturated Designs Given a Large-Dimensional Dataset

The problem of variable selection is fundamental to statistical modelling in diverse fields of sciences. In this paper, we study in particular the problem of selecting important variables in regression problems in the case where observations and labels of a real-world dataset are available. At first, we examine the performance of several existing statistical methods for analyzing a real large trauma dataset which consists of 7000 observations and 70 factors, that include demographic, transport and intrahospital data. The statistical methods employed in this work are the nonconcave penalized likelihood methods (SCAD, LASSO, and Hard), the generalized linear logistic regression, and the best subset variable selection (with AIC and BIC), used to detect possible risk factors of death. Supersaturated designs (SSDs) are a large class of factorial designs which can be used for screening out the important factors from a large set of potentially active variables. This paper presents a new variable selection approach inspired by supersaturated designs given a dataset of observations. The merits and the effectiveness of this approach for identifying important variables in observational studies are evaluated by considering several two-levels supersaturated designs, and a variety of different statistical models with respect to the combinations of factors and the number of observations. The derived results are encouraging since the alternative approach using supersaturated designs provided specific information that are logical and consistent with the medical experience, which may also assist as guidelines for trauma management.

Response modelling approach to robust parameter design methodology using supersaturated designs

ABSTRACT In recent years, both robust parameter designs (RPDs) and supersaturated designs (SSDs) have attracted a great deal of attention. In the present article, a combination of the above two techniques is considered. More precisely, we propose a construction of an effective SSD along with an analysis method in order to deal with the significant problem of the robust parameter design methodology (RPDM). Combining iterative Sure Independence Screening (SIS) variable selection and a penalized method, namely smoothly clipped absolute deviation (SCAD), we perform an analysis of the SSDs developed in the present work. The proposed methodology is applied in different models so as to show its effectiveness in many different scenarios, assuming both first-and second-order models of a response surface design. Two illustrative examples as well as numerous numerical experiments are conducted for plenty of cases. The results imply that the proposed method is highly effective for identifying the active effects of main factors, two-factor interactions, three-factor interactions, and the pure quadratic ones, under the assumption of effect sparsity.

Sure independence screening for real medical Poisson data

ABSTRACT The statistical modeling of big data bases constitutes one of the most challenging issues, especially nowadays. The issue is even more critical in case of a complicated correlation structure. Variable selection plays a vital role in statistical analysis of large data bases and many methods have been proposed so far to deal with the aforementioned problem. One of such methods is the Sure Independence Screening which has been introduced to reduce dimensionality to a relatively smaller scale. This method, though simple, produces remarkable results even under both ultra high dimensionality and big scale in terms of sample size problems. In this paper we dealt with the analysis of a big real medical data set assuming a Poisson regression model. We support the analysis by conducting simulated experiments taking into consideration the correlation structure of the design matrix.

A method for analyzing supersaturated designs inspired by control charts

Abstract The identification of active effects in supersaturated designs (SSDs) constitutes a problem of considerable interest to both scientists and engineers. The complicated structure of the design matrix renders the analysis of such designs a complicated issue. Although several methods have been proposed so far, a solution to the problem beyond one or two active factors seems to be inadequate. This article presents a heuristic approach for analyzing SSDs using the cumulative sum control chart (CUSUM) under a sure independence screening approach. Simulations are used to investigate the performance of the method comparing the proposed method with other well-known methods from the literature. The results establish the powerfulness of the proposed methodology.

Screening Active Effects in Supersaturated Designs with Binary Response via Control Charts

Supersaturated designs are designs in which the number of factors exceeds the run size; consequently, there are not enough degrees of freedom to estimate all the main effects. The goal here is to identify the dominant factors that constitute a small proportion of the overall set of factors, according to the assumption of effect sparsity. The analysis of such designs constitutes a challenging task and, even though many methods have been proposed in the literature assuming a normal response, only few works attempted to address the case of non-normal responses. In this paper, we propose a method for screening out the most important features in supersaturated designs assuming a Bernoulli distributed response. This new approach is based on an effective chart in Statistical Process Control, the cumulative sum control chart, combined with an information theoretic measure, and it is referred as the MIC algorithm. We judge the value of MIC through comparisons with three existing approaches suggested in the literature: the least absolute shrinkage and selection operator penalization method, and two feature selection algorithms, the Conditional Mutual Information Maximization and the minimal-redundancy-maximal-relevance. The simulation study reveals that the proposed method can be considered an advantageous method because of its extremely good performance in terms of statistical power. Copyright

Proximal support vector machine techniques on medical prediction outcome

ABSTRACT One of the major issues in medical field constitutes the correct diagnosis, including the limitation of human expertise in diagnosing the disease in a manual way. Nowadays, the use of machine learning classifiers, such as support vector machines (SVM), in medical diagnosis is increasing gradually. However, traditional classification algorithms can be limited in their performance when they are applied on highly imbalanced data sets, in which negative examples (i.e. negative to a disease) outnumber the positive examples (i.e. positive to a disease). SVM constitutes a significant improvement and its mathematical formulation allows the incorporation of different weights so as to deal with the problem of imbalanced data. In the present work an extensive study of four medical data sets is conducted using a variant of SVM, called proximal support vector machine (PSVM) proposed by Fung and Mangasarian [9]. Additionally, in order to deal with the imbalanced nature of the medical data sets we applied both a variant of SVM, referred as two-cost support vector machine and a modification of PSVM referred as modified PSVM. Both algorithms incorporate different weights one for each class examples.

A comparative study of the use of large margin classifiers on seismic data

In this work we present a study on the analysis of a large data set from seismology. A set of different large margin classifiers based on the well-known support vector machine (SVM) algorithm is used to classify the data into two classes based on their magnitude on the Richter scale. Due to the imbalance of nature between the two classes reweighing techniques are used to show the importance of reweighing algorithms. Moreover, we present an incremental algorithm to explore the possibility of predicting the strength of an earthquake with incremental techniques.

An orthogonal arrays approach to robust parameter designs methodology

Robust parameter design methodology was originally introduced by Taguchi [14] as an engineering methodology for quality improvement of products and processes. A robust design of a system is one in which two different types of factors are varied; control factors and noise factors. Control factors are variables with levels that are adjustable, whereas noise factors are variables with levels that are hard or impossible to control during normal conditions, such as environmental conditions and raw-material properties. Robust parameter design aims at the reduction of process variation by properly selecting the levels of control factors so that the process becomes insensitive to changes in noise factors. Taguchi [14 15] proposed the use of crossed arrays (inner–outer arrays) for robust parameter design. A crossed array is the cross-product of an orthogonal array (OA) involving control factors (inner array) and an OA involving noise factors (outer array). Objecting to the run size and the flexibility of crossed arrays, several authors combined control and noise factors in a single design matrix, which is called a combined array, instead of crossed arrays. In this framework, we present the use of OAs in Taguchis methodology as a useful tool for designing robust parameter designs with economical run size.