Delta plot analyses do not reflect individual differences in selective inhibition during picture-word interference tasks
Delta plot analyses do not reflect individual differences in selective inhibition during picture-word interference tasks Pamela Fuhrmeister* Audrey Bürki Department of Linguistics University of Potsdam Karl-Liebknecht-Straße 24-25 14476 Potsdam, Germany *Corresponding author, email: [email protected]
ELTA PLOT ANALYSES AND SELECTIVE INHIBITION Abstract Individuals vary in the time needed to produce a spoken word, for example when naming a picture. Similarly, they vary in how much their responses are delayed by interfering stimuli. Several studies have linked this variability to individual differences in inhibition abilities. The present study focuses on the relationship between the semantic interference effect in a picture-word naming task (longer picture-naming latencies in trials with semantically related than unrelated distractor words) and the change in the magnitude of this effect in the tail of the response time distribution. The positive correlation between these two measures, when computed separately for each participant, has been argued to reflect the involvement of selective inhibition in the naming task. We report a series of Bayesian meta-analyses to investigate the reliability of this relationship and to rule out possible alternative explanations. We find that while the relationship is robust, the same relationship is found when we calculate the index of inhibition for individual items, rather than participants. This latter finding challenges the assumption that this measurement reflects individual differences in inhibition ability.
Keywords : picture-word-interference task, semantic interference effect, selective inhibition, delta plot analyses, individual differences
ELTA PLOT ANALYSES AND SELECTIVE INHIBITION Delta plot analyses do not reflect individual differences in selective inhibition during picture-word interference tasks There is growing evidence that individuals differ in the time it takes to produce a spoken word and that these differences may be systematic rather than just noise in the data (e.g., Jongman et al., 2015; Laganaro et al., 2012; Shao et al., 2012). Specifically, some studies have found that various tests of cognitive skills predict individual differences in production latencies. The present study focuses on the role of inhibition in word production. The cognitive processes underlying word production are often assessed using picture-naming tasks. In one such task, the picture-word interference task, participants are asked to name a picture in the presence of a superimposed distractor word. A common finding is that naming latencies are longer when the superimposed distractor word is semantically related to the picture to be named than when the distractor is unrelated to the picture (the semantic interference effect, e.g., Lupker, 1979; see Bürki et al., 2020 for a review and meta-analysis). Individuals also vary in the magnitude of the semantic interference effect. Shao et al. (2012) found that a measure of non-selective inhibition (i.e., the ability to inhibit any irrelevant or unwanted response), predicted mean naming latencies on a picture-naming task. In Shao et al. (2013), they found a similar pattern using a picture-word interference task. On the other hand, in our own work, we failed to find evidence supporting the correlation between performance in picture naming or picture-word interference tasks and performance in several tasks measuring inhibition skills (e.g., Fuhrmeister et al., submitted). However, correlations between performance in independent tasks might not be the easiest way to demonstrate the link between linguistic and non-linguistic functions. Participants likely fluctuate in their performance within and between tasks for reasons that are independent of their domain-
ELTA PLOT ANALYSES AND SELECTIVE INHIBITION general abilities. This within-participant variability may make it hard to detect systematic differences across speakers. Notably, Shao et al. (2013) further demonstrated that the magnitude of a participant’s semantic interference effect was predicted by a measure of within-task selective inhibition (the ability to inhibit a specific response) using the delta plot procedure (e.g., Ridderinkhof et al., 2004), as described in the next paragraph. This latter finding is important, both for theoretical and methodological reasons. From a theoretical point of view, it provides the first evidence to date that the processes involved in language production recruit selective inhibition. Methodologically, it introduces a simple and straightforward procedure to test the involvement of inhibition skills in language production tasks, which paves the way for more fine-grained investigations. The advantage of this procedure (over correlations of performance across tasks for instance) is that inhibition and linguistic processes are measured in the same task. As a consequence, potential sources of noise are drastically reduced. One downside of this procedure however is that it potentially entails a certain circularity because the measure of selective inhibition is calculated using reaction time measurements of a subset of trials that are used to calculate the magnitude of the semantic interference effect. The present study has two aims: The first is to test the reliability of this finding. The second is to consider – and possibly rule out – alternative explanations. We address these two aims with a series of Bayesian-meta-analyses involving over 50 datasets. Distributional analyses of reaction time data
The delta plot procedure is a distributional analysis of reaction time data. It has often been used to measure selective inhibition on conflict tasks such as the Stroop task, Simon task, or Flanker task (e.g., Ridderinkhof et al., 2004, 2005; see also van den Wildenberg et al., 2010
ELTA PLOT ANALYSES AND SELECTIVE INHIBITION for review). In these types of tasks, participants have to respond to a stimulus on each trial, and some trials provide congruent information, and some provide incongruent information that needs to be inhibited. For example, in a Flanker task, participants see a line of arrows on the screen and are asked to indicate the direction of the middle arrow. The direction of the surrounding arrows can either be pointing in the same direction as the middle arrow (>>>>>, a congruent trial) or in the opposite direction (<<><<, an incongruent trial, e.g., Eriksen & Eriksen, 1974). Congruent trials are typically responded to faster than incongruent trials because incongruent trials contain information that needs to be inhibited. According to the activation suppression hypothesis (Ridderinkhof et al., 2004, 2005), inhibition takes time to build up, making it more effective in slower trials (trials with slower reaction times) than faster trials. With no inhibition applied, interference or congruency effects (i.e., the difference in reaction times between incongruent and congruent trials) should be larger at slower reaction times. However, when inhibition is applied during the task, the difference between congruent and incongruent trials tends to decrease with increasing reaction times and can even become negative (i.e., a facilitation effect). To derive a measure of inhibition using the delta plot procedure, reaction times per participant and per condition (i.e., congruent and incongruent) are rank ordered and divided into quantiles (i.e., percentile bins). Mean reaction times are computed for each condition in each quantile in order to calculate the difference between the two conditions (delta) for each quantile. The difference in effect sizes (i.e., slope) is calculated for the last delta segment (the segment between the slowest two quantiles). A positive slope for the last delta segment suggests that the congruency or interference effect got bigger as a function of reaction time, meaning that little to no inhibition was applied. If the last delta segment levels off or is even negative, this suggests that stronger inhibition was applied. Conceptually, this measure of inhibition can be understood as the effect of congruence or interference as a function of reaction time. This ELTA PLOT ANALYSES AND SELECTIVE INHIBITION measurement (the slope of the slowest delta segment) has been used in several studies as a measure of inhibition in different tasks and in different populations. For example, Ridderinkhof et al., (2005) tested children with and without ADHD on the Flanker task and found that a group of typically developing children’s slope of the last delta segment leveled off more and earlier compared to a group of children with ADHD, who were assumed to have a deficit in inhibition. As discussed briefly above, a handful of studies have applied this procedure to measure individual or group differences in inhibition deployed in the picture-word-interference task when pictures are presented with semantically related or unrelated distractor words. For example, Shao et al. (2013) found that a participant’s slope of the slowest delta segment was correlated with the magnitude of the semantic interference effect: Participants who applied more selective inhibition (i.e., had a less positive slope for the slowest delta segment) showed a smaller semantic interference effect overall. This finding was replicated in Shao et al. (2015), and they also extended this finding to another, similar task (the semantic blocking task). Roelofs et al. (2011) found a similar pattern in bilingual participants, in that participants with smaller semantic interference effects applied more inhibition on a picture-word-interference task, as indexed by the slope of the slowest delta segment. There may, however, be some alternative explanations for this pattern of results. For example, several previous studies have conducted distributional analyses of the semantic interference effect, and these studies all agree on the observation that the semantic interference increases with quantile (i.e., increasing reaction times; Bürki, submitted; Roelofs & Piai, 2017). Some of these studies even find that the effect is restricted to the slowest quantiles (Scaltritti et al., 2015). The distributional pattern of the semantic interference effect has been related to attention (lapses of attention generate both longer naming times and greater interference from the distractor word, De Jong, 1999; see also discussion in Roelofs, ELTA PLOT ANALYSES AND SELECTIVE INHIBITION Current study
The present study reports a series of meta-analyses (see Table 1 for a summary). In all the studies included in the meta-analysis, participants performed a picture-word interference task: They named pictures while ignoring distractors that were either of the same semantic category as the picture or unrelated. Our first meta-analysis aimed to test the reliability of the link between the magnitude of participants’ semantic interference effects and inhibition as measured by the slope of the slowest delta plot segment (i.e., the 4 th and 5 th quintiles, or 20% bins, e.g., Ridderinkhof et al., 2005), to assess the effect size of this correlation, as well as the uncertainty around that effect. Our second meta-analysis tested the relationship between the magnitude of the semantic interference effect and the first (i.e., fastest) delta segment. Shao et al. (2015) found that the slope of the fastest delta plot segment was not correlated with the magnitude of the semantic interference effect in one of two picture-word-interference experiments (and in two experiments using a semantic blocking task). The fact that they saw a relationship with the slope of the slowest delta segment but not with the fastest (with the exception of the one ELTA PLOT ANALYSES AND SELECTIVE INHIBITION experiment) lends support to claims made by the activation suppression hypothesis by Ridderinkhof et al. (2004)—that inhibition takes time to build up and is mostly reflected in the slope of the last (i.e., slowest) delta segment. Assuming however that the semantic interference effect increases with response times, a relationship with the slope of the fastest segment could be expected but with a smaller effect size. Our second meta-analysis tests the robustness of that effect and provides information on its size and precision. Having assessed the reliability of these effects, we examined the relationship between quintile and semantic interference. We are particularly interested in knowing whether the semantic interference effect increases between the 4 th and 5 th quintile, but we extend the analysis to other quintiles. We examine whether the effect is present in each quintile and whether it increases with quintile. If the effect mostly arises in the last quintiles or increases with quintiles, then we would expect a correlation between the magnitude of interference and slope of the slowest delta segment even if participants do not vary in their ability to deploy inhibition. Finally, we examined the relationship between the semantic interference effect size and slope of the slowest and fastest delta segments, this time computing quintiles over items rather than participants. If the correlation between the magnitude of interference and slope of the last two quintiles reflects inter-individual differences in the deployment of selective inhibition, we expect no correlation between semantic interference and the slope between the last two quintiles when we compute quintiles by items . Effect of interest Quintiles calculated by participants or items
1. Correlation between the magnitude of the semantic interference effect and the slowest delta plot segment Participants 2. Correlation between the magnitude of the semantic interference effect and the fastest delta plot segment Participants
ELTA PLOT ANALYSES AND SELECTIVE INHIBITION
3. Magnitude of the semantic interference effect in first quintile 4. Magnitude of the semantic interference effect in second quintile 5. Magnitude of the semantic interference effect in third quintile 6. Magnitude of the semantic interference effect in fourth quintile 7. Magnitude of the semantic interference effect in fifth quintile Participants 8. Correlation between the magnitude of the semantic interference effect and the slowest delta plot segment Items 9. Correlation between the magnitude of the semantic interference effect and the fastest delta plot segment Items
Table 1. Summary of meta-analyses: effects of interest and whether quintiles were calculated from participant or item data.
Methods
Data set
We worked with a subset of the data collected in a previous meta-analysis of the semantic interference effect (Bürki et al., 2020). We selected all the studies for which we had the raw data (a response time for each trial) and that had a stimulus onset asynchrony (SOA) between -160 and 160 ms. Several studies have reported semantic interference effects in this SOA range (e.g., Damian & Martin, 1999; Glaser & Düngelhoff, 1984 ; Starreveled & La Heij, 1996), a pattern supported by a recent meta-analysis (Bürki et al., 2020). We included two additional data sets that were recently collected from our lab. Participants in all these studies were adult native speakers of the language being tested, and they did not have language disorders. Languages tested in the various studies included German, English, French, Italian, Dutch, Spanish, and Mandarin. Only trials with distractor items that were semantically related or unrelated to the target picture were considered. Multiple experiments within a paper were treated as independent studies. Experiments where the same items were tested at different SOAs or with and without familiarization were split to generate one dataset for each level of these variables. The resulting dataset included a total of 54 experiments. More details on the studies can be found in Appendix C.
ELTA PLOT ANALYSES AND SELECTIVE INHIBITION Delta plot analyses
We used the delta plot procedure (e.g., Ridderinkhof et al., 2004) to calculate a measure of selective inhibition. This is a measure of an interference effect (here, the difference in reaction times for semantically related and unrelated trials) as a function of reaction time. We can use this to measure whether an interference effect increases or decreases with increasing reaction times. To do this, reaction time data is first separated by participant and condition (in our case, semantically related or unrelated trials). Reaction times are then sorted and divided into quantiles and the mean difference between conditions for each quantile is computed. We used five quintiles (i.e., 20% bins) as in Shao et al. (2013, 2015). The slope is then calculated for the slowest delta segment (i.e., the slope between quintiles four and five). We followed the procedure in, for example, De Jong (1994), Ridderinkhof et al. (2004), Roelofs et al., (2011), and Shao et al. (2013) to calculate the slope as follows: slope(quintile 4, quintile 5) = delta(quintile 5) - delta(quintile 4)/mean(quintile 5) - mean(quintile 4) Only correct responses were included in the delta slope calculations. A total of nine participants from all data sets were eliminated because they did not have enough data points to calculate quantiles. We also calculated the slope for the fastest delta segment (i.e., the slope between the first two quintiles) using the same procedure and used this measure to predict the magnitude of the semantic interference effect. If inhibition is reflected in the slope of the slowest delta segment, we would expect no relationship (or at least a much smaller relationship) between the slope of the fastest delta plot segment and the semantic interference effect.
ELTA PLOT ANALYSES AND SELECTIVE INHIBITION Finally, we repeated this procedure to calculate the delta slopes and mean semantic interference effects by items, rather than participants.
Analysis
Extraction of estimates for correlational analyses with delta plot segments.
For each study, we computed the correlation between the slope of the slowest delta segment and the mean semantic interference effect and also for the slope of the fastest delta segment and the mean semantic interference effect. We then used the Fisher z-transformation to transform correlation coefficients ( r values) to z values (Fisher, 1915) using the FisherZ function from the DescTools package (Signorell et al., 2020) in R. The z-transformed scores and their estimated standard error were entered into the meta-analyses described below. This process was repeated for by-item analyses. Extraction of estimates for distributional analyses.
To test whether the semantic interference effect increases as response times increase (or is only reflected in slower response times), we first computed five quintiles (20% bins) for each participant and each condition (semantically related and unrelated) separately. For each study, we fit a linear mixed effects model using the lme4 package (Bates et al., 2015) in R (R Core Team, 2020). Each model predicted naming latencies (the dependent variable) and included fixed effects of quintile (1-5), and condition (deviation coded, semantically related = .5, semantically unrelated = -.5), which was nested within quintile. Nested fixed effects allow us to test for simple effects (Schad et al., 2019), and in this case, we were interested in testing the difference in reaction times between semantically related and unrelated conditions at each level of the factor quintile (i.e., in each quintile separately). Random effects included by-participant random intercepts and slopes for quintile and by-participant random intercepts and slopes for condition, which were
ELTA PLOT ANALYSES AND SELECTIVE INHIBITION nested within quintile. Item random effects included by-item random intercepts and slopes for quintile and random intercepts and slopes for condition, which were nested within quintile. Correlations between random effects were set to zero. Meta-analyses.
Meta analyses estimate the size and precision of an effect in question from the effect sizes and standard errors of individual studies. In the present study, we tested the following effects in several different meta-analyses (see Table 1 for summary): the by-participant correlation between the slope of the slowest delta segment and the semantic interference effect, the by-participant correlation between the slope of the fastest delta segment and the semantic interference effect, the magnitude of the semantic interference effect in each quintile, and the correlations between the slope of the slowest (and fastest) delta segments and the semantic interference effect when quintiles were calculated over items. Both fixed-effects and random-effects meta-analyses can be performed, but they make different assumptions. Fixed-effects meta-analyses assume that all studies have the same true effect (e.g., Chen & Peace, 2013), but random-effects meta-analyses assume that the different studies have different true effects i (e.g., Sutton & Abrams, 2001). Each of the studies included in our data set were performed in different languages and in different labs; therefore, we assume a different underlying effect for each study and thus performed a random-effects meta-analysis. We assumed the following for each meta-analysis of correlations (note that the meta-analysis is performed on the Fisher z-transformed correlations: assumptions and prior pertain to the z-transformed score): Each study i has a true correlation of 𝜌 i that is normally distributed with a mean of 𝜌 and variance of = 10 . The observed correlation 𝑟 (cid:3036) in each study is assumed ELTA PLOT ANALYSES AND SELECTIVE INHIBITION to stem from a normal distribution with mean 𝜌 (cid:3036) and variance 𝜎 (cid:3036)(cid:2870) , the true standard error of the study. Details of the model specifications can be found in Equations (1). 𝑟 (cid:3036) | 𝜌 (cid:3036) , 𝜎 (cid:3036)(cid:2870) ~ 𝑁(cid:3435) 𝜌 (cid:3036) , 𝜎 (cid:3036)(cid:2870) (cid:3439) 𝑖 = 1, … , 𝑛, 𝜌 (cid:3036) | 𝜌 , 𝜏 (cid:2870) ~𝑁( 𝜌 , 𝜏 (cid:2870) ), 𝜌 ~ 𝑁 (
0, 10 ) , 𝜏 ~ 𝑁(0, 10), 𝜏 > 0 r i represents the observed correlation in each study i ; 𝜌 is the true correlation estimated by the model; 𝜎 (cid:3036)(cid:2870) represents the standard error for this study; and 𝜏 (cid:2870) represents the between-study variance. For meta-analyses testing the magnitude of the semantic interference effect in each quintile, we made the following assumptions: Each study i has a true effect of i that is normally distributed with a mean of and variance of = 100 . The observed effect of the predictor 𝑦 (cid:3036) in each study is assumed to stem from a normal distribution with mean 𝜃 (cid:3036) and variance 𝜎 (cid:3036)(cid:2870) , the true standard error of the study. Details of the model specifications can be found in Equations (2). 𝑦 (cid:3036) |𝜃 (cid:3036) , 𝜎 (cid:3036)(cid:2870) ~ 𝑁(cid:3435)𝜃 (cid:3036) , 𝜎 (cid:3036)(cid:2870) (cid:3439) 𝑖 = 1, … , 𝑛, 𝜃 (cid:3036) |𝜃, 𝜏 (cid:2870) ~𝑁(𝜃, 𝜏 (cid:2870) ), 𝜃 ~ 𝑁(0, 100 (cid:2870) ), 𝜏 ~ 𝑁(0, 100), 𝜏 > 0 y i represents the observed effect of the predictor in each study i ; is the true effect of the predictor estimated by the model; 𝜎 (cid:3036)(cid:2870) represents the variance for study i , estimated from the standard error of the effect of the predictor for this study; and 𝜏 (cid:2870) represents the between-study variance. For the intercept and standard deviation for meta-analyses of correlations, we chose weakly informative priors from a normal distribution with a mean of 0 and standard deviation of 10. For the standard deviation, we chose weakly informative priors from a normal distribution with a mean of 0 and standard deviation of 10. For the meta-analyses testing the magnitude of the semantic interference effect in each quintile, we chose weakly informative (1) (2) ELTA PLOT ANALYSES AND SELECTIVE INHIBITION priors from a normal distribution with a mean of 0 and a standard deviation of 100. For the standard deviation, we chose weakly informative priors from a normal distribution with a mean of 0 and standard deviation of 100. We additionally did sensitivity analyses for each meta-analysis to test whether the effects were robust with different priors. Effect sizes did not change with different priors for any of the meta-analyses, and details on sensitivity analyses can be found in Appendix B. Meta-analyses were performed in R (R Core Team, 2020) with the brms package (Bürkner, 2018). Data and analysis code can be found at https://osf.io/v2fx5/. Results
Results of the meta-analyses are summarized in Tables 2-4. Meta-analytic estimates, tau parameters, and their 95% credible intervals (CrI) are reported.
Meta-analysis: by-participant analyses Estimate CrI tau CrI
Correlation between slowest delta plot segment slope and semantic interference effect .52 [.44, .61] .23 [.16, .31] Correlation between fastest delta plot segment slope and semantic interference effect .18 [.11, .24] .13 [.04, .22]
Table 2. Results of meta-analyses testing the relationships between the semantic interference effect and the slowest and fastest delta plot segments when quintiles are calculated by participant. Estimates for correlational meta-analyses are Fisher z-transformed units; however, r and Fisher z-transformed values are very similar for r values between -.5 and .5 (see Figure 1).
ELTA PLOT ANALYSES AND SELECTIVE INHIBITION Figure 1. Posterior distributions of meta-analytic estimates (Fisher z-transformed correlations) of the relationship between A. the semantic interference effect and the slope of the fastest delta segment and B. the semantic interference effect and the slope of the slowest delta segment. Quintiles were calculated from participant data.
Meta-analysis: by-participant analyses Estimate 95% CrI tau 95% CrI
Semantic interference effect in quintile 1 6 ms [3, 9] 2 ms [0, 5] Semantic interference effect in quintile 2 11 ms [8, 14] 6 ms [3, 9] Semantic interference effect in quintile 3 17 ms [13, 21] 10 ms [7, 14] Semantic interference effect in quintile 4 27 ms [22, 32] 15 ms [11, 20] Semantic interference effect in quintile 5 49 ms [39, 58] 22 ms [12, 32]
Table 3. Results of meta-analyses testing the magnitude of the semantic interference effect per quintile (see Figure 2).
Figure 2. Posterior distributions of meta-analytic estimates of the semantic interference effect in each quintile. Quintiles were calculated by participant.
Meta-analysis: by-item analyses Estimate CrI tau CrI
Correlation between slowest delta plot segment slope and semantic interference effect .49 [.43, .54] .07 [.00, .15] Correlation between fastest delta plot segment slope and semantic interference effect .31 [.26, .36] .04 [.00, .12]
Table 4. Results of meta-analyses testing the relationships between the semantic interference effect and the slowest and fastest delta plot segments when quintiles are calculated by item. D en s i t y Correlation between the slope of the fastestdelta segment and the semantic interference effect A Correlation between the slope of the slowestdelta segment and the semantic interference effect B Estimate D en s i t y Quintile 1 A Quintile 2 B Quintile 3 C Quintile 4 D Quintile 5 E Estimate (ms)
ELTA PLOT ANALYSES AND SELECTIVE INHIBITION Estimates for correlational meta-analyses are Fisher z-transformed units; however, r and Fisher z-transformed values are very similar for r values between -.5 and .5 (see Figure 3).
Figure 3. Posterior distributions of meta-analytic estimates (Fisher z-transformed correlations) of the relationship between A. the semantic interference effect and the slope of the fastest delta segment and B. the semantic interference effect and the slope of the slowest delta segment. Quintiles were calculated by items rather than participants.
Discussion
Many studies of word production report a wide range of individual variability in the time needed to produce a word, for example, in a picture-naming task (e.g., Jongman et al., 2015; Laganaro et al., 2012; Shao et al., 2012). Several studies have attributed individual differences in naming times or in the magnitude of the semantic interference effect (e.g., in the picture-word-interference task) to individual differences in inhibition. In conflict tasks such as the picture-word-interference task, inhibition is often measured by the slope of the slowest delta segment using the delta plot procedure (Roelofs et al., 2011; Shao et al., 2013, 2015). A common finding is that the slope of the slowest delta segment predicts the magnitude of the semantic interference effect for individual participants (Shao et al., 2013, 2015). The present study reports a series of Bayesian meta-analyses that included 54 different datasets. The first meta-analysis aimed to provide an estimate of the correlation between D en s i t y Correlation between fastest delta plot slopeand semantic interference (items) A Correlation between slowest delta plot slopeand semantic interference (items) B Estimate
ELTA PLOT ANALYSES AND SELECTIVE INHIBITION inhibition (as measured by the slope of the slowest delta segment for each participant) and the magnitude of the semantic interference effect at the individual level and to obtain a more precise estimate of the effect. Results from that meta-analysis suggest that this effect is reliable and robust: a participant’s slope of the slowest delta segment is positively correlated with the magnitude of the semantic interference effect for this participant. This is in line with the hypothesis that participants who apply less inhibition during the task as indexed by a steeper, more positive slope, show larger semantic interference effects. According to the activation suppression hypothesis, inhibition takes time to build up, which is why the slope of the slowest delta segment is typically used to index inhibition (e.g., Ridderinkhof et al., 2004). Although inhibition can be applied earlier and can be reflected in earlier delta segments, it should be most apparent in the slowest segment (e.g., van den Wildenberg et al., 2010). Shao et al. (2015) additionally tested the relationship between the fastest delta segment slope and the semantic interference effect and found that these were significantly correlated in one of two experiments using the picture-word-interference task, but not in either experiment using the semantic blocking task. They argue that because they found a significant correlation with the slowest delta segment and the magnitude of the semantic interference effect in all four experiments but only found the relationship between the fastest segment and the semantic interference effect in one of the four experiments, this suggests that the slowest delta segment is indeed an index of inhibition. In contrast with this claim, we found a positive relationship between the slope of the fastest delta segment and the semantic interference effect, though it was smaller than the size of the effect for the slowest delta segment. It is possible that with higher-powered studies or achieving higher power by means of a meta-analysis, we will see that delta slopes start leveling off much earlier. Our findings are not entirely inconsistent with the activation suppression hypothesis, in that the effect size of the slowest delta segment slope was larger than the fastest. However, if ELTA PLOT ANALYSES AND SELECTIVE INHIBITION inhibition takes time to build up, it is unexpected that we would still see this positive relationship between the slope of the fastest delta segment and the semantic interference effect. Thirdly, we replicate the finding that the semantic interference effect increases with response times. Several mechanisms have been put forward to explain this relationship, for example, fluctuations of attention (Roelofs & Piai, 2017), selection of wrong responses (Van Maanen & Van Rijn, 2008) or differences in temporal alignment between the processing of the distractor and the encoding of the target word (Bürki, submitted). Irrespective of the underlying mechanisms, the distributional pattern of the semantic interference effect makes it plausible to consider an alternative, mechanistic explanation to the relationship between the slope between the last two (and first two) quintiles and participants’ speed. According to this account, this relationship does not (only) reflect the deployment of inhibition but is a by-product of the general relationship between response times and semantic interference. As this effect becomes greater between the last two quintiles, participants who show less of an effect overall are expected to show less of an increase in the last two quintiles, irrespective of whether they deploy inhibition or not. To further address this issue, we conducted the same correlational meta-analyses, but we calculated quintiles by item instead of by participant. If the slope of the slowest delta segment is indeed a measure of individual differences in the deployment of inhibition, we should not see a relationship with the mean semantic interference effect and either of the delta plot segment slopes (fastest or slowest) if we calculated these by items, rather than by participants. Instead, we found that there were indeed robust relationships between these slopes and the mean semantic interference effect, and these effect sizes are similar (descriptively) to the participant analyses. These results lend further support to the idea that the relationship between the slope of the last delta segment and the mean semantic ELTA PLOT ANALYSES AND SELECTIVE INHIBITION interference effect is a by-product of the relationship between response times and the semantic interference effect and cannot be assumed to reflect individual differences in selective inhibition ability. Though we argue that the slope of the slowest delta segment does not index individual differences in selective inhibition for the semantic interference effect, we cannot rule out the possibility that this measure indeed indexes inhibition in other tasks. For example, Shao et al. (2015) found a relationship between the slowest delta segment slope and the semantic interference effect in both a picture-word-interference task and a semantic blocking task. In the semantic blocking task, participants name pictures in small sets. Some sets include pictures from the same semantic category, and some contain pictures from various semantic categories. For the semantic blocking task, they did not find a relationship between the slope of the fastest delta segment and the mean semantic interference effect, and they replicated this finding in a second experiment. Thus, it is possible that the magnitude of the semantic interference effect only increases with increasing reaction times in certain tasks, such as the picture-word-interference task, and that the delta plot procedure may be a reliable measure of individual differences in selective inhibition for other tasks. Shao et al. (2015) also did not find a similar pattern with the Stroop task, in which participants name a color and have to ignore printed distractor words that are names of colors. Therefore, it is possible that the slope of the slowest delta segment as a measure of inhibition is specific to the task. However, future work will need to test which tasks this measure can be applied to. The findings in the present study, however, challenge the view that individual differences in selective inhibition in picture-word-interference tasks can be captured in the slope of the slowest delta segment. ELTA PLOT ANALYSES AND SELECTIVE INHIBITION Acknowledgements The authors would like to thank all the authors who shared their datasets. This research was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – project number 317633480 – SFB 1287, Project B05 (Bürki).
ELTA PLOT ANALYSES AND SELECTIVE INHIBITION References Aristei, S., & Abdel Rahman, R. (2013). Semantic interference in language production is due to graded similarity, not response relevance.
Acta Psychologica , (3), 571–582. https://doi.org/10.1016/j.actpsy.2013.09.006 Aristei, S., Melinger, A., & Abdel Rahman, R. (2011). Electrophysiological chronometry of semantic context effects in language production. Journal of Cognitive Neuroscience , (7), 1567–1586. https://doi.org/10.1162/jocn.2010.21474 Bates, D. Maechler, M., Bolker, B., & Walker, S. (2015). Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software, 67(1), 1-48. doi:10.18637/jss.v067.i01. Bürki, A., Elbuy, S., Madec, S., & Vasishth, S. (2020). What did we learn from forty years of research on semantic interference? A Bayesian meta-analysis. Journal of Memory and Language , , 104125. Bürki, A. (submitted). Picture-word interference in language production studies: Attention and processing times matter. Bürkner, P. C. (2018). Advanced Bayesian Multilevel Modeling with the R Package brms. The R Journal, 10(1), 395–411. Chen, D. G. D., & Peace, K. E. (2013). Applied meta-analysis with R . CRC Press. Cutting, J. C., & Ferreira, V. S. (1999). Semantic and phonological information flow in the production lexicon.
Journal of Experimental Psychology: Learning, Memory, and Cognition , (2), 318. Damian, M. F., & Martin, R. C. (1999). Semantic and phonological codes interact in single word production. Journal of Experimental Psychology. Learning, Memory, and Cognition, 25 (2), 345–361. https://doi.org/10.1037//0278-7393.25.2.345 Damian, M. F., & Bowers, J. S. (2003). Locus of semantic interference in picture-word interference tasks.
Psychonomic Bulletin & Review , (1), 111–117. https://doi.org/10.3758/BF03196474 Damian, M. F., & Spalek, K. (2014). Processing different kinds of semantic relations in picture-word interference with non-masked and masked distractors. Frontiers in Psychology , . https://doi.org/10.3389/fpsyg.2014.01183 De Jong, R., Berendsen, E., & Cools, R. (1999). Goal neglect and inhibitory limitations: Dissociable causes of interference effects in conflict situations. Acta psychologica , (2-3), 379-394. De Jong, R., Liang, C. C., & Lauber, E. (1994). Conditional and unconditional automaticity: a dual-process model of effects of spatial stimulus-response correspondence. Journal of Experimental Psychology: Human Perception and Performance , (4), 731. de Zubicaray, G. I., Hansen, S., & McMahon, K. L. (2013). Differential processing of thematic and categorical conceptual relations in spoken word production. Journal of Experimental Psychology. General , (1), 131–142. https://doi.org/10.1037/a0028717 Eriksen, B. A., & Eriksen, C. W. (1974). Effects of noise letters upon the identification of a target letter in a nonsearch task. Perception & psychophysics , (1), 143-149. ELTA PLOT ANALYSES AND SELECTIVE INHIBITION Fisher, R. A. (1915). Frequency distribution of the values of the correlation coefficient in samples from an indefinitely large population.
Biometrika , (4), 507-521. Finocchiaro, C., & Navarrete, E. (2013). About the locus of the distractor frequency effect: Evidence from the production of clitic pronouns. Journal of Cognitive Psychology , (7), 861–872. https://doi.org/10.1080/20445911.2013.832254 Fuhrmeister, P., Madec, S., Lorenz, A., Elbuy, S., & Bürki, A. (submitted). Behavioral and EEG evidence for inter-individual variability in late encoding stages of word production. Gauvin, H. S., Jonen, M. K., Choi, J., McMahon, K., & Zubicaray, G. I. de. (2018). No lexical competition without priming: Evidence from the picture–word interference paradigm: Quarterly Journal of Experimental Psychology . https://doi.org/10.1177/1747021817747266 Glaser, W. R., & Düngelhoff, F.-J. (1984). The time course of picture-word interference . Journal of Experimental Psychology: Human Perception and Performance, 10 (5), 640–654. https://doi.org/10.1037/0096-1523.10.5.640 Hartendorp, M. O., Van der Stigchel, S., & Postma, A. (2013). To what extent do we process the nondominant object in a morphed figure? Evidence from a picture–word interference task.
Journal of Cognitive Psychology , (7), 843-860. Hutson, J., & Damian, M. F. (2014). Semantic gradients in picture-word interference tasks: Is the size of interference effects affected by the degree of semantic overlap? Frontiers in Psychology , . https://doi.org/10.3389/fpsyg.2014.00872 Janssen, N., Schirm, W., Mahon, B. Z., & Caramazza, A. (2008). Semantic interference in a delayed naming task: Evidence for the response exclusion hypothesis. Journal of Experimental Psychology. Learning, Memory, and Cognition , (1), 249–256. https://doi.org/10.1037/0278-7393.34.1.249 Jongman, S. R., Roelofs, A., & Meyer, A. S. (2015). Sustained attention in language production: An individual differences investigation. Quarterly Journal of Experimental Psychology , (4), 710-730. Laganaro, M., Valente, A., & Perret, C. (2012). Time course of word production in fast and slow speakers: a high density ERP topographic study. NeuroImage , (4), 3881-3888. Lupker, S. J. (1979). The semantic nature of response competition in the picture-word interference task. Memory & Cognition , (6), 485–495. https://doi.org/10.3758/BF03198265 Mädebach, A., Oppermann, F., Hantsch, A., Curda, C., & Jescheniak, J. D. (2011). Is there semantic interference in delayed naming? Journal of Experimental Psychology. Learning, Memory, and Cognition , (2), 522–538. https://doi.org/10.1037/a0021970 Piai, V., Roelofs, A., Jensen, O., Schoffelen, J.-M., & Bonnefond, M. (2014). Distinct Patterns of Brain Activity Characterise Lexical Activation and Competition in Spoken Word Production. PLOS ONE , (2), e88674. https://doi.org/10.1371/journal.pone.0088674 Piai, V., Roelofs, A., & Schriefers, H. (2012). Distractor strength and selective attention in picture-naming performance. Memory & Cognition , ELTA PLOT ANALYSES AND SELECTIVE INHIBITION Ridderinkhof, K. R., Scheres, A., Oosterlaan, J., & Sergeant, J. A. (2005). Delta plots in the study of individual differences: new tools reveal response inhibition deficits in AD/Hd that are eliminated by methylphenidate treatment.
Journal of abnormal psychology , (2), 197. Ridderinkhof, K. R., van den Wildenberg, W. P., Wijnen, J., & Burle, B. (2004). Response inhibition in conflict tasks is revealed in delta plots. Cognitive neuroscience of attention , , 377. Rodríguez-Ferreiro, J., Davies, R., & Cuetos, F. (2014). Semantic domain and grammatical class effects in the picture–word interference paradigm. Language, Cognition and Neuroscience , (1), 125–135. https://doi.org/10.1080/01690965.2013.788195 Roelofs, A. (2008). Dynamics of the attentional control of word retrieval: Analyses of response time distributions. Journal of Experimental Psychology: General , (2), 303. Roelofs, A., & Piai, V. (2017). Distributional analysis of semantic interference in picture naming. Quarterly Journal of Experimental Psychology , (4), 782-792. Roelofs, A., Piai, V., & Garrido Rodriguez, G. (2011). Attentional inhibition in bilingual naming performance: evidence from delta-plot analyses. Frontiers in psychology , , 184. Sailor, K., Brooks, P. J., Bruening, P. R., Seiger-Gardner, L., & Guterman, M. (2009). Exploring the time course of semantic interference and associative priming in the picture–word interference task: Quarterly Journal of Experimental Psychology . https://doi.org/10.1080/17470210802254383 Scaltritti, M., Navarrete, E., & Peressotti, F. (2015). Distributional analyses in the picture–word interference paradigm: Exploring the semantic interference and the distractor frequency effects.
The Quarterly Journal of Experimental Psychology , (7), 1348–1369. https://doi.org/10.1080/17470218.2014.981196 Schad, D. J., Vasishth, S., Hohenstein, S., & Kliegl, R. (2020). How to capitalize on a priori contrasts in linear (mixed) models: A tutorial. Journal of Memory and Language , , 104038. Shao, Z., Roelofs, A., & Meyer, A. S. (2012). Sources of individual differences in the speed of naming objects and actions: The contribution of executive control. Quarterly Journal of Experimental Psychology , (10), 1927-1944. Shao, Z., Meyer, A. S., & Roelofs, A. (2013). Selective and nonselective inhibition of competitors in picture naming. Memory & cognition , (8), 1200-1211. Shao, Z., Roelofs, A., Martin, R. C., & Meyer, A. S. (2015). Selective inhibition and naming performance in semantic blocking, picture-word interference, and color–word Stroop tasks. Journal of Experimental Psychology: Learning, Memory, and Cognition , (6), 1806. Signorell, A. et mult. al. (2020). DescTools: Tools for descriptive statistics. R package version 0.99.38. Sutton, A. J., & Abrams, K. R. (2001). Bayesian methods in meta-analysis and evidence synthesis. Statistical methods in medical research , (4), 277-303. ELTA PLOT ANALYSES AND SELECTIVE INHIBITION Starreveld, P. A., & La Heij, W. (1996). Time-course analysis of semantic and orthographic context effects in picture naming.
Journal of Experimental Psychology: Learning, Memory, and Cognition, 22 (4), 896–918. https://doi.org/10.1037/0278-7393.22.4.896 Van Den Wildenberg, W. P., Wylie, S. A., Forstmann, B. U., Burle, B., Hasbroucq, T., & Ridderinkhof, K. R. (2010). To head or to heed? Beyond the surface of selective action inhibition: a review.
Frontiers in human neuroscience , , 222. Van Maanen, L., & Van Rijn, H. (2008). The picture-word interference effect is a Stroop effect after all. In Proceedings of the Annual Meeting of the Cognitive Science Society (Vol. 30, No. 30). Vieth, H. E., McMahon, K. L., & de Zubicaray, G. I. (2014). The roles of shared vs. Distinctive conceptual features in lexical access.
Frontiers in Psychology , . https://doi.org/10.3389/fpsyg.2014.01014 Zhang, Q., Feng, C., Zhu, X., & Wang, C. (2016). Transforming semantic interference into facilitation in a picture–word interference task. Applied Psycholinguistics , (5), 1025-1049. ELTA PLOT ANALYSES AND SELECTIVE INHIBITION Appendix A. In addition to doing meta-analyses on correlations between the slopes of the delta plot segments and the mean semantic interference effect, we tested an interaction effect of the delta slope and the semantic interference effect in mixed effects models. This enables us to include random effects in the models and therefore generalize to participants and items not included in the current sample. The assumptions and priors were the same as indicated for mixed effects models above. We first fit separate linear mixed effects models for each study. Models were fit using the lme4 package (Bates et al., 2015) in R (R Core Team, 2018). Each model predicted naming latencies (the dependent variable) and included fixed effects of condition (deviation coded, semantically related = .5, semantically unrelated = -.5), the slope of the slowest delta plot segment, and their interaction. Random effects included by-participant random intercepts and slopes for condition, random slopes for the slowest delta plot segment, and their interaction, as well as by-item random intercepts and slopes for condition, random slopes for the slowest delta plot segment, and their interaction. Correlations between random effects were set to zero. This procedure was repeated to test for the interaction between the semantic interference effect and the slope of the fastest delta plot segment. All fixed and random effects in these models were otherwise identical to the previous ones. Variance inflation factors were calculated and suggested that multicollinearity was not an issue. Results are summarized in the table below. The pattern of results is similar to the correlational meta-analyses when testing the interaction effect.
Meta-analysis: Interactions Estimate 95% CrI tau 95% CrI
Interaction between slowest delta plot segment slope and semantic interference effect 37 ms [31, 43] 12 ms [4, 19] Interaction between fastest delta plot segment slope and semantic interference effect 15 ms [8, 21] 7 ms [.30, 17]
ELTA PLOT ANALYSES AND SELECTIVE INHIBITION Appendix B. Sensitivity analyses. 1. Correlation between the magnitude of the semantic interference effect and the slowest delta plot segment (participant analysis)
Prior Estimate 95% CrI tau 95% CrI
Uniform (-3,3) .52 [.44, .60] .23 [.16, .31] Uniform (-10,10) .52 [.43, .60] .23 [.16, .31] 2. Correlation between the magnitude of the semantic interference effect and the fastest delta plot segment (participant analysis)
Prior Estimate 95% CrI tau 95% CrI
Uniform (-3,3) .18 [.11, .24] .14 [.04, .22] Uniform (-10,10) .18 [.10, .25] .14 [.03, .22] 3. Magnitude of the semantic interference effect in first quintile
Prior Estimate 95% CrI tau 95% CrI
Normal (0,200) 6 ms [3, 9] 2 ms [0, 5] Uniform (-200,200) 6 ms [3, 9] 2 ms [0, 5] Normal (0,50) 6 ms [3, 9] 2 ms [0, 5] 4. Magnitude of the semantic interference effect in second quintile
Prior Estimate 95% CrI tau 95% CrI
Normal (0,200) 11 ms [8, 14] 6 ms [3, 9] Uniform (-200,200) 11 ms [8, 14] 6 ms [3, 9] Normal (0,50) 11 ms [8, 14] 6 ms [3, 9] 5. Magnitude of the semantic interference effect in third quintile
Prior Estimate 95% CrI tau 95% CrI
Normal (0,200) 17 ms [13, 21] 10 ms [7,14] Uniform (-200,200) 17 ms [13, 21] 10 ms [7,14] Normal (0,50) 17 ms [13, 21] 10 ms [7,14] 6. Magnitude of the semantic interference effect in fourth quintile
Prior Estimate 95% CrI tau 95% CrI
Normal (0,200) 27 ms [22, 32] 15 ms [11, 20] Uniform (-200,200) 27 ms [22, 32] 15 ms [11, 20] Normal (0,50) 27 ms [21, 32] 15 ms [11, 20] 7. Magnitude of the semantic interference effect in fifth quintile
Prior Estimate 95% CrI tau 95% CrI
Normal (0,200) 49 ms [39, 58] 21 ms [12, 32]
ELTA PLOT ANALYSES AND SELECTIVE INHIBITION Uniform (-200,200) 49 ms [39, 58] 22 ms [12, 32] Normal (0,50) 48 ms [39, 58] 21 ms [12, 32] 8. Correlation between the magnitude of the semantic interference effect and the slowest delta plot segment (item analysis)
Prior Estimate 95% CrI tau 95% CrI
Uniform (-3,3) .49 [.43, .54] .07 [.01, .15] Uniform (-10,10) .49 [.43, .54] .07 [.00, .15] 9. Correlation between the magnitude of the semantic interference effect and the fastest delta plot segment (item analysis)
Prior Estimate 95% CrI tau 95% CrI
Uniform (-3,3) .31 [.26, .37] .04 [.00, .12] Uniform (-10,10) .31 [.26, .37] .04 [.00, .12]
ELTA PLOT ANALYSES AND SELECTIVE INHIBITION Appendix C. Studies used in meta-analyses.
Experiments Reference
Aristei.2011 Aristei et al. (2011) Aristei.2013 Aristei & Abdel Rahman (2013) Cutting.1999.1 Cutting & Ferreira (1999) Cutting.1999.2 Cutting.1999.3a.1 Cutting.1999.3a.2 Damian.2003.SOA-100 Damian & Bowers (2003) Damian.2003.SOA0 Damian.2003.SOA100 Damian.2014 Damian & Spalek (2014) deZubicaray.2013 De Zubicaray et al. (2013) Finocchiaro.2013.1 Finocchiaro & Navarrete (2013) Fuhrmeister.unpublished - Fuhrmeister2.unpublished - Gauvin.2018.1.fam Gauvin et al. (2018) Gauvin.2018.1.nofam Gauvin.2018.2.fam Gauvin.2018.2.nofam Hartendorp.2013.1 Hartendorp et al. (2013) Hartendorp.2013.2 Hutson.2014.1 Hutson & Damian (2014) Hutson.2014.2 Janssen.2008.1a Janssen et al. (2008) Janssen.2008.2a Maedebach.2011.2 Mädebach et al. (2011) Maedebach.2011.4 Maedebach.2011.5a Maedebach.2011.6 Piai.2012 Piai et al. (2012) Piai.2012.2 Piai.2014 Piai et al. (2014) Piai.unpublished - Python.unpublished - Rodriguez.2014 Rodríguez-Ferreiro et al. (2014) Roelofs.2008.3 Roelofs (2008) Sailor.2009.1.SOA-150 Sailor et al. (2009) Sailor.2009.1.SOA150 Sailor.2009.2.SOA0 Scaltritti.2015.1 Scaltritti et al. (2015) Scaltritti.2015.3 Shao.2015.Exp.1 Shao et al. (2015) Shao.2015.Exp.2 Starreveld.2013.1.SOA0 Starreveld et al. (2013) Starreveld.2013.1.SOA43 Starreveld.2013.1.SOA86 Starreveld.2013.1.SOAminus43
ELTA PLOT ANALYSES AND SELECTIVE INHIBITION29