Vicenç Quera

Detecting group differences in sequential association using sampled permutations: Log odds, kappa, and phi compared

When determining whether a particular transition is more characteristic of one group than of another, two things are required: an index associated with the transition of interest and a statistical test that can determine whether group membership systematically affects values for that index. Here the familiar parametrict test is compared with a test based on sampled permutations. Indices considered are the odds and log odds ratio, Yule’sQ, Wampold’s (1989) transformed kappa, and phi. The odds and log odds ratio are monotonically increasing functions of Yule’sQ and so give similar results. Yule’sQ and phi are essentially rank order invariant and usually give similar results. Transformed kappa, however, rank orders subjects somewhat differently than the others; moreover, it appears somewhat biased. With respect to the tests, when subjects are 20 or more it does not matter much whether sampled permutation or parametrict tests are used; both yield essentially the same result. However, when subjects are fewer than 20, or whenever there is any other reason to think that parametric assumptions may not be met, permutation tests are recommended. A computer program that effects such tests is described.

SDIS: A sequential data interchange standard

Few general-purpose computer programs are available that analyze sequential categorical data. If there were a sequential data interchange standard—a standard way of representing sequential data—then it would be more attractive to write general-purpose computer programs for such data Moreover, interlaboratory sharing would be facilitated. The present paper defines such a standard, called the sequential data interchange standard, or SDIS. Both the SDIS data language and a parsing program for data that follow SDIS conventions are described. The parsing program will be made available to researchers who wish to develop analysis programs for sequential data

Testing sequential association: Estimating exact p values using sampled permutations.

Assigning p values to all transitions in a matrix based on their z scores is problematic on 2 counts: The z scores may not be normally distributed, and transitions are interconnected. Permutation tests, which require far fewer assumptions, are an attractive alternative to the standard asymptotic methods for assigning significance. Moreover, when asymptotic z scores are only somewhat above their critical value and sequences are short, often the exact probabilities of permutation tests are not less than .05. Log-linear and permutation methods may be used to winnow the set of transitions initially identified as significant even further. A computer program that performs these tests is available from the authors.

Observer agreement for timed-event sequential data: A comparison of time-based and event-based algorithms

Observer agreement is often regarded as the sine qua non of observational research. Cohen’s κ ?is a widely used index and is appropriate when discrete entities—such as a turn-of-talk or a demarcated time interval—are presented to pairs of observers to code. κ-like statistics and agreement matrices are also used for the timed-event sequential data produced when observers first segment and then code events detected in the stream of behavior, noting onset and offset times. Such κs are of two kinds: time-based and event-based. Available for download is a computer program (OASTES; Observer Agreement for Simulated Timed Event Sequences) that simulates the coding of observers of a stated accuracy and then computes agreement statistics for two time-based κs (with and without tolerance) and three event-based κs (one implemented in The Observer, one in INTERACT, and one in GSEQ). On the basis of simulation results presented here, and due to the somewhat different information provided by each, the reporting of both a time-based and an event-based κ?is recommended.

Observer agreement for event sequences: Methods and software for sequence alignment and reliability estimates

When sequences of discrete events, or other units, are independently coded by two coders using a set of mutually exclusive and exhaustive codes, but the onset times for the codes are not preserved, it is often unclear how pairs of protocols should be aligned. Yet such alignment is required before Cohen’s kappa, a common agreement statistic, can be computed. Here we describe a method—based on the Needleman and Wunsch (1970) algorithm originally devised for aligning nucleotide sequences—for optimally aligning such sequences; we also offer the results of a simulation study of the behavior of alignment kappa with a number of variables, including number of codes, varying degrees of observer accuracy, sequence length, code variability, and parameters governing the alignment algorithm. We conclude that (1) under most reasonable circumstances, observer accuracies of 90% or better result in alignment kappas of .60 or better; (2) generally, alignment kappas are not strongly affected by sequence length, the number of codes, or the variability in the codes’ probability; (3) alignment kappas are adversely affected when missed events and false alarms are possible; and (4) cost matrices and priority orders used in the algorithm should favor substitutions (i.e., disagreements) over insertions and deletions (i.e., missed events and false alarms). Two computer programs were developed: Global Sequence Alignment, or GSA, for carrying out the simulation study, and Event Alignment, or ELign, a user-oriented program that computes alignment kappa and provides the optimal alignment given a pair of event sequences.

Randomization Tests for Systematic Single Case Designs Are Not Always Appropriate

The authors used a Monte Carlo simulation to examine how the violation of the exchangeability assumption affects empirical Type I error rates of the LMH randomization test (J. R. Levin, L. A. Marascuilo, & L. J. Hubert, 1978). Simulation results showed that the LMH test is not always an appropriate technique for analyzing systematic designs when data are autocorrelated. The use of both conditional and unconditional randomization distributions is proposed as a way of synthesizing the literature.

OTS: A program for converting Noldus Observer data files to SDIS files

A program for converting Noldus Observer data files (ODF) to sequential data interchange standard (SDIS) files is described. Observer users who convert their data files can then take advantage of various flexible and powerful data modification and computational procedures available in the Generalized Sequential Querier, a program that assumes SDIS-formatted files.

ANSEC: A BASIC package for lag sequential analysis of observational data

Three BASIC programs for processing observational, nonconcurrent sequential data are presented. The programs follow Sackett’s lag sequential analysis method and have the innovations of running on interactional microcomputers and of providing plots of results. The outcome of the analysis is stored on a magnetic disk, facilitating a further application of probabilistic models to the transformed data. The Allison-Liker correction for the comparison test between expected and observed lag probabilities is included in the programs.

Coding interactions in Motivational Interviewing with computer-software

Display Omitted We present advantages of software support in coding Motivational Interviewing.The software instrument showed excellent to good interrater reliability.The software and paper instrument obtained similar results on convergent validity.Behavior slicing indicated that it is not necessary to code the entire interaction.We provide three coded demo interactions in two languages for the software version. Motivational Interviewing (MI) is an evidence-based behavior change intervention. The interactional change processes that make MI effective have been increasingly studied using observational coding schemes. We introduce an implementation of a software-supported MI coding scheme-the Motivational Interviewing Treatment Integrity code (MITI)-and discuss advantages for process researchers. Furthermore, we compared reliability of the software version with prior results of the paper version. A sample of 14 double-coded dyadic interactions showed good to excellent interrater reliabilities. We selected a second sample of 22 sessions to obtain convergent validity results of the software version: substantial correlations were obtained between the software instrument and the Rating Scales for the Assessment of Empathic Communication. Finally, we demonstrate how the software version can be used to test whether single code frequencies obtained by using intervals shorter than 20min (i.e., 5 or 10min) are accurate estimates of the respective code frequencies for the entire session (i.e., behavior slicing). Our results revealed that coding only a 10-min interval provides accurate estimates of the entire session. Our study demonstrates that the software implementation of the MITI is a reliable and valid instrument. We discuss advantages of the software version for process research in MI.

Determining shoal membership: A comparison between momentary and trajectory-based methods

Miller and Gerlai proposed two methods for determining shoal membership in Danio rerio, one based on momentary mean inter-individual distances and the other on post hoc analysis of the trajectories of nearest-neighbor distances. We propose a method based on momentary nearest-neighbor distances and compare the three methods using simulation. In general, our method yielded results that were more similar to their second method than their first one, and is computationally simpler.