Benjamin Scheibehenne

Testing Adaptive Toolbox Models : A Bayesian Hierarchical Approach

Many theories of human cognition postulate that people are equipped with a repertoire of strategies to solve the tasks they face. This theoretical framework of a cognitive toolbox provides a plausible account of intra- and interindividual differences in human behavior. Unfortunately, it is often unclear how to rigorously test the toolbox framework. How can a toolbox model be quantitatively specified? How can the number of toolbox strategies be limited to prevent uncontrolled strategy sprawl? How can a toolbox model be formally tested against alternative theories? The authors show how these challenges can be met by using Bayesian inference techniques. By means of parameter recovery simulations and the analysis of empirical data across a variety of domains (i.e., judgment and decision making, childrens cognitive development, function learning, and perceptual categorization), the authors illustrate how Bayesian inference techniques allow toolbox models to be quantitatively specified, strategy sprawl to be contained, and toolbox models to be rigorously tested against competing theories. The authors demonstrate that their approach applies at the individual level but can also be generalized to the group level with hierarchical Bayesian procedures. The suggested Bayesian inference techniques represent a theoretical and methodological advancement for toolbox theories of cognition and behavior.

An Introduction to Bayesian Hypothesis Testing for Management Research

In management research, empirical data are often analyzed using p-value null hypothesis significance testing (pNHST). Here we outline the conceptual and practical advantages of an alternative analysis method: Bayesian hypothesis testing and model selection using the Bayes factor. In contrast to pNHST, Bayes factors allow researchers to quantify evidence in favor of the null hypothesis. Also, Bayes factors do not require adjustment for the intention with which the data were collected. The use of Bayes factors is demonstrated through an extended example for hierarchical regression based on the design of an experiment recently published in the Journal of Management. This example also highlights the fact that p values overestimate the evidence against the null hypothesis, misleading researchers into believing that their findings are more reliable than is warranted by the data.

Bayesian Evidence Synthesis Can Reconcile Seemingly Inconsistent Results: The Case of Hotel Towel Reuse

Recent concerns that psychological science may suffer from a lack of replicability have prompted a methodological reorientation that values preregistration of hypotheses and data-analysis plans, high statistical power, exact replications, and the assessment of cumulative knowledge through meta-analysis (Eerland, Sherrill, Magliano, & Zwaan, 2016; Open Science Collaboration, 2015). This reorientation raises the question of how exactly new and old findings ought to be combined. Here, we outline a Bayesian approach that updates knowledge about an effect as new studies become available. This method—Bayesian evidence synthesis—affords several advantages: It provides a continuous measure of evidence that indexes the degree of support for the null hypothesis versus an alternative hypothesis (Monden et al., in press), it distinguishes between evidence for the absence of an effect versus absence of evidence for an effect (e.g., Dienes, 2014), and it allows a continual updating of knowledge as new studies appear, indefinitely and without a sampling plan or stopping rule (e.g., Rouder, 2014). Below, we highlight these advantages using a concrete example concerning the effectiveness of descriptive social norms in facilitating ecological behavior. Descriptive social norms indicate which behavior is typical or normal in a given situation (Cialdini, Reno, & Kallgren, 1990). Such information can influence people’s behavior in important ways (P. W. Schultz, Nolan, Cialdini, Goldstein, & Griskevicius, 2007). In a widely cited study on the effectiveness of descriptive social norms (Goldstein, Cialdini, & Griskevicius, 2008), two groups of hotel guests received different messages that encouraged them to reuse their towels. One message simply informed the guests about the benefits of environmental protection (the control condition), and the other message indicated that the majority of guests actually reused their towels in the past (the descriptive-social-norm condition). The results suggested that the latter message facilitated towel reuse (Experiment 1—descriptive-social-norm condition: 44.1% reuse, control condition: 35.1% reuse; p = .05; Experiment 2—descriptive-social-norm conditions (combined): 44.5% reuse, control condition: 37.2% reuse; p = .03). A search across all studies in the literature that cited this original publication and a separate search combining the terms “social norm” and “towel reuse” revealed five replication experiments that assessed the proportion of hotel guests who reused their towels, with a total sample size of 2,466 participants (Bohner & Schlüter, 2014; Mair & Bergin-Seers, 2010; W. P. Schultz, Khazian, & Zaleski, 2008). All five experiments arguably failed to replicate the original finding (all ps > .14). However, this apparent contradiction can be resolved by a Bayesian reanalysis. In the first step of this reanalysis,1 we recorded how many participants reused their towel in each of the two conditions in all seven experiments. Next, for each experiment, we obtained a separate one-sided Bayes factor for a test of equality of two proportions (e.g., Gunel & Dickey, 1974; Jamil, Marsman, Ly, Morey, & Wagenmakers, in press; Jeffreys, 1961). In this analysis, the null hypothesis was that the proportions of guests who did and did not reuse their towels are equal, whereas the default alternative hypothesis was that the proportions are independent and uniformly distributed between 0 and 1, with the added restriction that the proportion in the 644081 PSSXXX10.1177/0956797616644081Scheibehenne et al.Bayesian Evidence Synthesis research-article2016

Using Bayesian hierarchical parameter estimation to assess the generalizability of cognitive models of choice

To be useful, cognitive models with fitted parameters should show generalizability across time and allow accurate predictions of future observations. It has been proposed that hierarchical procedures yield better estimates of model parameters than do nonhierarchical, independent approaches, because the formers’ estimates for individuals within a group can mutually inform each other. Here, we examine Bayesian hierarchical approaches to evaluating model generalizability in the context of two prominent models of risky choice—cumulative prospect theory (Tversky & Kahneman, 1992) and the transfer-of-attention-exchange model (Birnbaum & Chavez, 1997). Using empirical data of risky choices collected for each individual at two time points, we compared the use of hierarchical versus independent, nonhierarchical Bayesian estimation techniques to assess two aspects of model generalizability: parameter stability (across time) and predictive accuracy. The relative performance of hierarchical versus independent estimation varied across the different measures of generalizability. The hierarchical approach improved parameter stability (in terms of a lower absolute discrepancy of parameter values across time) and predictive accuracy (in terms of deviance; i.e., likelihood). With respect to test–retest correlations and posterior predictive accuracy, however, the hierarchical approach did not outperform the independent approach. Further analyses suggested that this was due to strong correlations between some parameters within both models. Such intercorrelations make it difficult to identify and interpret single parameters and can induce high degrees of shrinkage in hierarchical models. Similar findings may also occur in the context of other cognitive models of choice.

Constructing Preference from Experience: The Endowment Effect Reflected in External Information Search.

People often attach a higher value to an object when they own it (i.e., as seller) compared with when they do not own it (i.e., as buyer)--a phenomenon known as the endowment effect. According to recent cognitive process accounts of the endowment effect, the effect is due to differences between sellers and buyers in information search. Whereas previous investigations have focused on search order and internal search processes (i.e., in memory), we used a sampling paradigm to examine differences in search termination in external search. We asked participants to indicate selling and buying prices for monetary lotteries in a within-subject design. In an experience condition, participants had to learn about the possible outcomes and probabilities of the lotteries by experiential sampling. As hypothesized, sellers tended to terminate search after sampling high outcomes, whereas buyers tended to terminate search after sampling low outcomes. These differences in stopping behavior translated into samples of the lotteries that were differentially distorted for sellers and buyers; the amount of the distortion was predictive of the resulting size of the endowment effect. In addition, for sellers search was more extended when high outcomes were rare compared with when low outcomes were rare. Our results add to the increasing evidence that the endowment effect is due, in part, to differences in predecisional information search.

Selecting decision strategies: The differential role of affect

Many theories on cognition assume that people adapt their decision strategies depending on the situation they face. To test if and how affect guides the selection of decision strategies, we conducted an online study (N = 166), where different mood states were induced through video clips. Results indicate that mood influenced the use of decision strategies. Negative mood, in particular anger, facilitated the use of non-compensatory strategies, whereas positive mood promoted compensatory decision rules. These results are in line with the idea that positive mood broadens the focus of attention and thus increases the use of compensatory decision strategies that take many pieces of information into account, whereas negative mood narrows the focus of attention and thus fosters non-compensatory strategies that rely on a selective use of information. The results further indicate that gaining a deeper theoretical understanding of the cognitive mechanisms that govern decision processes requires taking emotions into account.

Psychophysiological arousal and inter- and intraindividual differences in risk-sensitive decision making

Abstract The current study assessed peripheral responses during decision making under explicit risk, and tested whether intraindividual variability in choice behavior can be explained by fluctuations in peripheral arousal. Electrodermal activity (EDA) and heart rate (HR) were monitored in healthy volunteers (N = 68) during the Roulette Betting Task. In this task, participants were presented with risky gambles to bet on, with the chances of winning varying across trials. Hierarchical Bayesian analyses demonstrated that EDA and HR acceleration responses during the decision phase were sensitive to the chances of winning. Interindividual differences in this peripheral reactivity during risky decision making were related to trait sensitivity to punishment and trait sensitivity to reward. Moreover, trial‐by‐trial variation in EDA and HR acceleration responses predicted a small portion of intraindividual variability in betting choices. Our results show that psychophysiological responses are sensitive to explicit risk and can help explain intraindividual heterogeneity in choice behavior.

Betting on Illusory Patterns: Probability Matching in Habitual Gamblers.

Abstract Why do people gamble? A large body of research suggests that cognitive distortions play an important role in pathological gambling. Many of these distortions are specific cases of a more general misperception of randomness, specifically of an illusory perception of patterns in random sequences. In this article, we provide further evidence for the assumption that gamblers are particularly prone to perceiving illusory patterns. In particular, we compared habitual gamblers to a matched sample of community members with regard to how much they exhibit the choice anomaly ‘probability matching’. Probability matching describes the tendency to match response proportions to outcome probabilities when predicting binary outcomes. It leads to a lower expected accuracy than the maximizing strategy of predicting the most likely event on each trial. Previous research has shown that an illusory perception of patterns in random sequences fuels probability matching. So does impulsivity, which is also reported to be higher in gamblers. We therefore hypothesized that gamblers will exhibit more probability matching than non-gamblers, which was confirmed in a controlled laboratory experiment. Additionally, gamblers scored much lower than community members on the cognitive reflection task, which indicates higher impulsivity. This difference could account for the difference in probability matching between the samples. These results suggest that gamblers are more willing to bet impulsively on perceived illusory patterns.

A generalized distance function for preferential choices

Many cognitive theories of judgement and decision making assume that choice options are evaluated relative to other available options. The extent to which the preference for one option is influenced by other available options will often depend on how similar the options are to each other, where similarity is assumed to be a decreasing function of the distance between options. We examine how the distance between preferential options that are described on multiple attributes can be determined. Previous distance functions do not take into account that attributes differ in their subjective importance, are limited to two attributes, or neglect the preferential relationship between the options. To measure the distance between preferential options it is necessary to take the subjective preferences of the decision maker into account. Accordingly, the multi-attribute space that defines the relationship between options can be stretched or shrunk relative to the attention or importance that a person gives to different attributes describing the options. Here, we propose a generalized distance function for preferential choices that takes subjective attribute importance into account and allows for individual differences according to such subjective preferences. Using a hands-on example, we illustrate the application of the function and compare it to previous distance measures. We conclude with a discussion of the suitability and limitations of the proposed distance function.

A hierarchical Bayesian model of the influence of run length on sequential predictions

Two models of how people predict the next outcome in a sequence of binary events were developed and compared on the basis of gambling data from a lab experiment using hierarchical Bayesian techniques. The results from a student sample (N = 39) indicated that a model that considers run length (“drift model”)—that is, how often the same event has previously occurred in a row—provided a better description of the data than did a stationary model taking only the immediately prior event into account. Both, expectation of negative and of positive recency was observed, and these tendencies mostly grew stronger with run length. For some individuals, however, the relationship was reversed, leading to a qualitative shift from expecting positive recency for short runs to expecting negative recency for long runs. Both patterns could be accounted for by the drift model but not the stationary model. The results highlight the importance of applying hierarchical analyses that provide both group- and individual-level estimates. Further extensions and applications of the approach in the context of the prediction literature are discussed.