Eugene S. Edgington

Customization of pain treatments: single-case design and analysis.

The aim of this paper is to acquaint pain researchers and practitioners with recent developments in the single-case experimental approach and their potential to allow for tailoring the treatment and its evaluation to the specific complaints, aptitudes, or profile of the individual patient, without violating the canons of good science and practice. After contrasting the single-case experimental approach and the case-study approach, we show the possibilities of customization in design, measurement, and test statistics. This is done by distinguishing 2 types of single-case designs--alternation designs and phase designs--and 2 types of replication strategies--simultaneous replications and sequential replications. In addition, tailor-made randomization tests are proposed for alternation, phase, and simultaneous replication designs and the combining of P values to perform a meta-analysis on designs that are sequentially replicated. With our emphasis on: 1) randomization in the design; 2) the possibilities for a statistical test (together with the determination of power and the calculation of effect sizes); 3) the importance of reliable and valid measurement; and 4) the role of replication, we demonstrate how internal validity, statistical-conclusion validity, construct validity, and external validity concerns can be dealt with within a single-case experimental approach framework. Finally, the many research examples and references to clinical work illustrate the usefulness of the approach.

Randomization Tests for One-Subject Operant Experiments

Summary If statistical tests are to be applied to data from one-subject experiments, those experiments, like ordinary multiple-subject experiments, should utilize a type of random sampling or random assignment that provides an appropriate basis for the determination of statistical significance. In one-subject operant experiments it is common to assign blocks of treatment times, rather than individual treatment times, to the treatments. This mode of assignment has made it difficult to justify the application of analysis of variance, time-series analysis, and other parametric procedures in such experiments. On the other hand, randomization tests whose validity for such application is assured can be readily developed. Procedures for developing and applying these randomization tests are given.

Validity of Randomization Tests for One-Subject Experiments.

Valid Statistical tests for one-subject experiments are necessary to justify Statistical inferences and to ensure the acceptability of research reports to a wide range of journals and readers. The validity of randomization tests for one-subject experiments is examined in this paper. A randomization test is a procedure for determining significance in the following manner. A test statistic (e.g.,t or F) is computed for a set of research data. The value of the test statistic is called the “obtained test statistic value.” The data are then divided repeatedly, and the test statistic is computed for each data division. If the proportion of the data divisions giving a test statistic value as large as the obtained test statistic value is no greater than α, the test statistic is significant at the α level. Any Statistical test is said to be a randomization test when the significance of its test statistic is determined by the randomization test procedure. Determination of significance by the randomization test procedure permits the valid application of any Statistical test, whether it be as simple as at test or as complex as factorial multivariate analysis of variance, for one-subject as well as multiple-subject experiments. For the randomization test procedure to be valid for a one-subject experiment, there must be random assignment of treatment times to treatments (i.e., random determination of when each treatment is to be given); specification, before observing the data, of the test statistic and the criterion to be employed for discarding data; and, in the determination of significance, division of the data in a manner consistent with the random assignment procedure.

Randomization Tests: Computer Time Requirements

Summary Even though randomization tests are the most powerful of nonparametric tests and are the only valid tests to employ when there has been random assignment, but not random selection, of subjects in experiments (a common practice in psychology), such tests are rarely used by psychologists. The limited adoption of randomization tests is primarily a consequence of the great amount of computation they require. The present study shows, however, that the computation for randomization test counterparts of the t test and one-way analysis of variance can be relatively inexpensive when performed by a high-speed computer.

Randomization tests for restricted alternating treatments designs

Alternating Treatments Designs (ATD) with random assignment of the treatments to the measurement times provide very powerful single-case experiments. However, complete randomization might cause too many consecutive administrations of the same treatment to occur in the design. In order to exclude these possibilities, an ATD with restricted randomization can be used. In this article we provide a general rationale for the random assignment procedure in such a Restricted Alternating Treatments Design (RATD), and derive the corresponding randomization test. A software package for randomization tests in RATD, ATD and other single-case experimental designs [Van Damme & Onghena Single-case randomization tests, version 1.1, Department of Psychology, Katholieke Universiteit Leuven, Belgium] is discussed.

Randomized single-subject experimental designs.

Books on single-subject methodology tend to focus on traditional operant research techniques and thus provide little or no discussion of random introduction of treatments and statistical tests based on such randomization, i.e. randomization tests. Those books are the principal references to which researchers must turn for a comprehensive coverage of single-subject methodology, and so many researchers are likely to be unaware of the relevance of randomization (random assignment of treatment times to treatments) and randomization tests to single-subject experimentation. That is unfortunate because randomization is necessary in order to draw valid statistical inferences about treatment effects. The role of randomization in providing control over major threats to internal validity is explained in this article, and a number of randomized single-subject designs and their applications are provided. Appropriate rank tests are specified, and sources of free software for other, more complex, statistical tests are given.

Overcoming Obstacles to Single-Subject Experimentation.

Kazdin (this issue) discussed two types of problems supposedly associated with the use of randomization tests for single-subject experiments: the random introduction of treatments and the repeated alternation of treatments. Both types of problems represent obstacles to single-subject experimentation, generally, not just to the determination of significance by the randomization test procedure. Ways to reduce the adverse effects associated with the random introduction of treatments and the repeated alternation of treatments are presented.

Combining Probabilities from Discrete Probability Distributions.

The combining of probabilities from separate studies has been discussed frequently in regard to probabilities based on continuous probability distributions. Little has been written, however, regarding the combining of probabilities from discrete distributions, such as probability distributions for nonparametric tests. This paper explains how to combine probabilities when some or all of them are from discrete probability distributions.

Randomization tests : application to single-cell and other single-unit neuroscience experiments

The application of randomization tests for statistical determination of the significance of experimental manipulations on single cells and other types of single units in neuroscience is described. Applications of standard parametric tests like analysis of variance (ANOVA) and t tests to data from single-subject experiments have been severely criticized for lack of validity and those criticisms are relevant to parametric statistical tests for data from other types of single-unit experiments. A broad class of statistical tests known as randomization tests, on the other hand, has been free of such criticism. Randomization tests have been applied to data from various types of single units in neuroscience, where their validity in the absence of random sampling makes them especially valuable. Until the advent of computers, the computational requirements of randomization tests rendered them impractical. Randomization test computer programs are now readily available. Procedures for access to a public domain program are given in the text.

A Computer Program for Pattern Analysis of Rod-and-Frame Tests

Pattern analysis for rod-and-frame tests has been developed to identify a subjects test score patterns in relation to hypothesized patterns. This task can be done by calculating p values as a measure of strength of a relationship between hypothesized patterns and test data. The p values are obtained by means of a special randomization correlation test. A FORTRAN program for this test is briefly described.