George A. Morgan

Measures of clinical significance.

Behavioral scientists are interested in answering three basic questions when examining the relationships between variables (Kirk, 2001). First, is an observed result real or should it be attributed to chance (i.e., statistical significance)? Second, if the result is real, how large is it (i.e., effect size)? Third, is the result large enough to be meaningful and useful (i.e., clinical or practical significance)? In this last column in the series, we treat clinical significance as equivalent to practical significance. Judgments by the researcher and the consumers (e.g., clinicians and patients) regarding clinical significance consider factors such as clinical benefit, cost, and side effects. Although there is no formal statistical test of clinical significance, researchers suggest using one of three types of effect size measures to assist in interpreting clinical significance. These include the strength of association between variables (r family effect size measures), the magnitude of the difference between treatment and comparison groups (d family effect size measures), and measures of risk potency. In this paper, we review the d and r effect size measures and five measures of risk potency: odds ratio, risk ratio, relative risk reduction, risk difference, and number needed to treat. Finally, we review a relatively new effect size, AUC (which for historical reasons irrelevant to the current discussion stands for area under the receiver operating characteristic [ROC] curve), that integrates many of the others and is directly related to clinical significance. Each of these measures, however, has limitations that require the clinician to be cautious about interpretation. Guidelines are offered to facilitate the interpretation and understanding of clinical significance. Problems With Statistical Significance

Impulse Buying Behavior of Apparel Purchasers

This study compared three samples offemale consumers (textiles and clothing [TC] and non-TC students and older non-student consumers) on four impulse buying dimensions and planned buying, other shopping behaviors, and demographic characteristics. The study also identified possible predictor variables of impulse buying. Non-student consumers were most likely to be planned buyers while students were most likely to be impulse buyers. Comparisons of the three groups of consumers on other shopping behaviors and demographic variables further supported the proposition that these groups made up different market segments. The TC students may represent young consumers especially interested in apparel. Multiple regression analyses revealed that impulse buying behavior could be predicted from other shopping behaviors and demographic variables, especially for the student groups. The findings provide a conceptual and empirical analysis of impulse buying and identify how specific variables are related to each of four dimensions of impulse buying.

Mastery motivation in infants and toddlers: Is it greatest when tasks are moderately challenging?☆

Abstract We examined the effects of the difficulty level of a task on two measures of mastery motivation: task persistence and task pleasure. Children of three age groups (12, 24, and 36 months) were given six puzzles of varying difficulty levels to complete. The results reveal that task persistence varied with the difficulty of the task. Infants and toddlers showed greater persistence at moderately challenging tasks as compared to difficult tasks. There was no effect of difficulty level upon degree of task pleasure; however, a significant increase in task pleasure did occur between 24 and 36 months. We hypothesized that the transition from sensorimotor to preoperational intelligence allows the infant to gain greater pleasure from the perception of his or her own effectance in acting upon the environment. Correlations between persistence and cognitive measures decreased with age, suggesting that motivation and cognition may become less interrelated with development. Implications for cognitive-motivational assessment are also discussed.

Null Hypothesis Significance Testing: Effect Size Matters

A statistically significant outcome only indicates that it is likely that there is a relationship between variables. It does not describe the extent (strength) of that relationship. In this article, emphasis is placed on the importance of assessing the strength of the relationship between the independent and dependent variables using effect size indices. Effect size indices for the d family and r family are introduced, along with formulas for their direct and indirect computation for both the t test and chi-square test. A subset of the variables and concepts examined in the Whittaker and Manfredo study are reported here to demonstrate why an effect size index should be computed. Statistical analyses (either t test or chi-square test) were performed on the original sample of 796 and three smaller sample sizes (398, 200, and 100) randomly selected from the initial sample. Effect size indices were computed for each statistical test. The results indicated that the size of the sample directly affects the t or chi-square statistic and p , but the effect size was independent of the sample size. Effect sizes should, therefore, accompany reported p values.

Data Collection Techniques

We have provided an overview of techniques used to assess variables in the applied behavioral sciences. Most of the methods are used by both quantitative/positivist and qualitative/constructivist researchers but to different extents. Qualitative researchers prefer more open-ended, less structured data collection techniques than do quantitative researchers. Direct observation of participants is common in experimental and qualitative research; it is less common in so-called survey research, which tends to use self-report questionnaires. It is important that investigators use instruments that are reliable and valid for the population and purpose for which they will be used. Standardized instruments have manuals that provide norms and indexes of reliability and validity. However, if the populations and purpose on which these data are based are different from yours, it may be necessary for you to develop your own instrument or provide new evidence of reliability and validity.

Logistic Regression and Discriminant Analysis: Use and Interpretation

Predicting the probability that an event will or will not occur, as well as identifying the variables useful in making the prediction, is important in the health sciences and is central to risk research. Two statistical techniques can be used appropriately to predict a dichotomous dependent variable: discriminant analysis and logistic regression. In our last column, we discussed linear regression, used when the dependent variable is continuous. Discriminant analysis can be used with a dichotomous dependent variable, but the method requires several assumptions for the predictions to be optimal. Grimm and Yarnold (1995) provide a nontechnical discussion of discriminant analysis and logistic regression. This column primarily focuses on logistic regression, which requires fewer assumptions than discriminant analysis. Even when the assumptions required for discriminant analysis are satisfied, logistic regression still performs well. In logistic regression, the probability of an event occurring is estimated. Logistic regression models can include one or more independent variables that may be either dichotomous or continuous. Logistic regression with one independent variable is called bivariate logistic regression; with two or more independent variables, logistic regression is called multiple logistic regression. These should not be confused with multinomial logistic regression where the dependent variable has more than two categories. In this column, the focus is on dichotomous outcomes of the dependent variable. In linear regression, the regression coefficient represents the

Single-Factor Repeated-Measures Designs: Analysis and Interpretation

In this column we discussed the selection and interpretation of appropriate statistical tests for single-factor within-subjects/ repeated-measures designs and provided an example from the literature. The parametric tests that we discussed were the t test for paired or correlated samples and the single-factor repeated-measures ANOVA. We also mentioned four nonparametric tests to be used in single-factor within-subjects/repeated-measures designs, but they are relatively rare in the literature. The Compton et al. (2001) article did not provide effect size measures, but they could be computed from the means and standard deviations. Remember that a statistically significant t or ANOVA (even ifp < .001) does not mean that there was a large effect, especially if the sample was large. In the Compton example, the sample was quite small (N = 14), and the findings do reflect a large effect size.

Maternal parenting stress and mothers' reports of their infants' mastery motivation.

Mastery motivation is a psychological force that stimulates an individual to attempt to master a task that is challenging to him or her. This prospective longitudinal study examined the relationship between maternal stress, using the Parenting Stress Index-Short Form, and infant mastery motivation, using the Dimensions of Mastery Questionnaire, for 150 mother-infant pairs assessed at both 6- and 18-months of age. Infants of mothers with elevated stress levels at 6 months tended to show lower mastery motivation at 18 months (standardized beta=-.46, p=.001). Conversely, infants with lower general competence (standardized beta=-.24, p=.021) and lower persistence during social interactions with other children (standardized beta=-.18, p=.037) at 6 months of age had mothers with elevated total stress at 18 months of age. Implications for programs which simultaneously intervene with child and mother are discussed.

Developmental Transformations in Mastery Motivation

Mastery motivation and similar concepts such as effectance motivation, intrinsic motivation, and competence motivation have roots in an appealing belief that there is an intrinsic motive to control the environment, to master skills, and to be effective. It is usually assumed that individual differences in this motivation will show some continuity across infancy and early childhood and will influence later competence.

Quasi-Experimental Designs

In earlier columns, we introduced 2 types of independent variables, active and attribute. The randomized experimental and quasi-experimental approaches both have an active or manipulated independent variable, which has 2 or more values, called levels. The dependent variable is the outcome measure or criterion of the study. In both randomized and quasi-experimental approaches, the active independent variable has at least 1 level that is some type of intervention or manipulation given to participants in the experimental group during the study. Usually there is also a comparison or control group, which is given another level of the independent variable. There can be more than 2 levels or groups (e.g., 2 or more different interventions plus one or more comparison groups), but for simplicity our examples will be limited to no more than 2 groups. As discussed in an earlier column, the key ctfference between quasi-experiments and randomized experiments is whether the participants are assigned randomly to the groups or levels of the independent variable. In quasi-experiments random assignment of the participants is not done; thus, the groups are probably nonequivalent, and there are usually alternative interpretations of the results that make definitive conclusions difficult. For example, if some children diagnosed with attention-deficit/ hyperactivity disorder were treated with stimulants and others were not, later differences between the groups could be due to many factors. Unless children were randomly assigned to receive the medication or not, the groups could be different because families who volunteer (or agree) to have their children medicated may be different, in important ways, from those who do not. Or, perhaps, the more disruptive kids were given stimulants. Thus, later problem behaviors (or positive outcomes) could be due to initial differences between the groups.