Andrew Brand

Accuracy of Effect Size Estimates from Published Psychological Research

A Monte-Carlo simulation was used to model the biasing of effect sizes in published studies. The findings from the simulation indicate that, when a predominant bias to publish studies with statistically significant results is coupled with inadequate statistical power, there will be an overestimation of effect sizes. The consequences such an effect size overestimation will then have on meta-analyses and power analyses are highlighted and discussed along with measures which can be taken to reduce the problem.

Multiple Trials May Yield Exaggerated Effect Size Estimates

ABSTRACT Published psychological research attempting to support the existence of small and medium effect sizes may not have enough participants to do so accurately, and thus, repeated trials or the use of multiple items may be used in an attempt to obtain significance. Through a series of Monte-Carlo simulations, this article describes the results of multiple trials or items on effect size estimates when the averages and aggregates of a dependent measure are analyzed. The simulations revealed a large increase in observed effect size estimates when the numbers of trials or items in an experiment were increased. Overestimation effects are mitigated by correlations between trials or items, but remain substantial in some cases. Some concepts, such as a P300 wave or a test score, are best defined as a composite of measures. Troubles may arise in more exploratory research where the interrelations among trials or items may not be well described.

More Voodoo Correlations: When Average-Based Measures Inflate Correlations

ABSTRACT A Monte-Carlo simulation was conducted to assess the extent that a correlation estimate can be inflated when an average-based measure is used in a commonly employed correlational design. The results from the simulation reveal that the inflation of the correlation estimate can be substantial, up to 76%. Additionally, data was re-analyzed from two previously published studies to determine the extent that the correlation estimate was inflated due to the use of an averaged based measure. The re-analyses reveal that correlation estimates had been inflated by just over 50% in both studies. Although these findings are disconcerting, we are somewhat comforted by the fact that there is a simple and easy analysis that can be employed to prevent the inflation of the correlation estimate that we have simulated and observed.

Assessing the Effects of Technical Variance on the Statistical Outcomes of Web Experiments Measuring Response Times

A simulation was conducted to assess the effect of technical variance on the statistical power of web experiments measuring response times. The results of the simulation showed that technical variance reduced the statistical power and the accuracy of the effect size estimate by a negligible magnitude. This finding therefore suggests that researchers’ preconceptions concerning the unsuitability of web experiments for conducting research using response time as a dependent measure are misguided.

Cognitive rehabiliation for Parkinson's disease demantia: a study protocol for a pilot randomised controlled trial

BackgroundThere is growing interest in developing non-pharmacological treatments to address the cognitive deficits apparent in Parkinson’s disease dementia and dementia with Lewy bodies. Cognitive rehabilitation is a goal-oriented behavioural intervention which focuses on improving everyday functioning through management of cognitive difficulties; it has been shown to be effective in Alzheimer’s disease. To date, no studies have assessed its potential efficacy for addressing the impact of cognitive impairment in people with Parkinson’s disease or dementia with Lewy bodies.Methods/designParticipants (n = 45) will be recruited from movement disorders, care for the elderly and memory clinics. Inclusion criteria include: a diagnosis of Parkinson’s disease, Parkinson’s disease dementia or dementia with Lewy bodies according to consensus criteria and an Addenbrooke’s Cognitive Examination – III score of ≤ 82. Exclusion criteria include: a diagnosis of any other significant neurological condition; major psychiatric disorder, including depression, which is not related to the patient’s Parkinson’s disease and unstable medication use for their physical or cognitive symptoms. A single-blind pilot randomised controlled trial, with concurrent economic evaluation, will compare the relative efficacy of cognitive rehabilitation with that of two control conditions. Following a goal-setting interview, the participants will be randomised to one of the three study arms: cognitive rehabilitation (eight weekly sessions), relaxation therapy (eight weekly sessions) or treatment as usual. Randomisation and treatment group allocation will be carried out by a clinical trials unit using a dynamic adaptive sequential randomisation algorithm. The primary outcomes are patients’ perceived goal attainment at a 2-months post-intervention assessment and a 6-months follow-up. Secondary outcomes include patients’ objective cognitive performance (on tests of memory and executive function) and satisfaction with goal attainment, carers’ perception of patients’ goal attainment and patients’ and carers’ health status and psychosocial well-being, measured at the same time points. Cost-effectiveness will be examined to explore the design of a larger cost-effectiveness analysis alongside a full trial.DiscussionThis pilot study will evaluate the application of cognitive rehabilitation for the management of cognitive difficulties associated with Parkinson’s disease dementia and dementia with Lewy bodies. The results of the study will inform the design of a fully powered randomised controlled trial.Trial registrationISRCTN16584442 DOI 10.1186/ISRCTN16584442 13 April 2015

A pilot randomized controlled trial of a self-management group intervention for people with early-stage dementia (The SMART study).

BACKGROUND Self-management equips people to manage the symptoms and lifestyle changes that occur in long-term health conditions; however, there is limited evidence about its effectiveness for people with early-stage dementia. This pilot randomized controlled trial (RCT) explored the feasibility of a self-management intervention for people with early-stage dementia. METHODS The participants were people with early-stage dementia (n = 24) and for each participant a caregiver also took part. Participants were randomly allocated to either an eight-week self-management group intervention or treatment as usual (TAU). Assessments were conducted at baseline, three months and six months post-randomization by a researcher blind to group allocation. The primary outcome measure was self-efficacy score at three months. RESULTS Thirteen people with dementia were randomized to the intervention and 11 to TAU. Two groups were run, the first consisting of six people with dementia and the second of seven people with dementia. There was a small positive effect on self-efficacy with the intervention group showing gains in self-efficacy compared to the TAU group at three months (d = 0.35), and this was maintained at six months (d = 0.23). In terms of intervention acceptability, attrition was minimal, adherence was good, and satisfaction ratings were high. Feedback from participants was analyzed with content analysis. The findings suggest the positive aspects of the intervention were that it fostered independence and reciprocity, promoted social support, offered information, and provided clinician support. CONCLUSIONS This study has provided preliminary evidence that self-management may be beneficial for people with early-stage dementia.

Alpha values as a function of sample size, effect size, and power: accuracy over inference.

Tables of alpha values as a function of sample size, effect size, and desired power were presented. The tables indicated expected alphas for small, medium, and large effect sizes given a variety of sample sizes. It was evident that sample sizes for most psychological studies are adequate for large effect sizes defined at .8. The typical alpha level of .05 and desired power of 90% can be achieved with 70 participants in two groups. It was perhaps doubtful if these ideal levels of alpha and power have generally been achieved for medium effect sizes in actual research, since 170 participants would be required. Small effect sizes have rarely been tested with an adequate number of participants or power. Implications were discussed.

Effectiveness of portable electronic and optical magnifiers for near vision activities in low vision: a randomised crossover trial

To compare the performance of near vision activities using additional portable electronic vision enhancement systems (p‐EVES), to using optical magnifiers alone, by individuals with visual impairment.

Portable electronic vision enhancement systems in comparison with optical magnifiers for near vision activities: an economic evaluation alongside a randomized crossover trial

To determine the incremental cost‐effectiveness of portable electronic vision enhancement system (p‐EVES) devices compared with optical low vision aids (LVAs), for improving near vision visual function, quality of life and well‐being of people with a visual impairment.

Acoustic prepulse inhibition: One ear is better than two, but why and when?

We examined whether monaural prepulses produce more prepulse inhibition (PPI) because they might be more attention capturing (unambiguous to locate) than binaural prepulses. Monaural and binaural PPI was tested under normal and verbal and visuospatial attention manipulation conditions in 55 healthy men, including 29 meditators. Attention manipulations abolished monaural PPI superiority, similarly in meditators and meditation-naïve individuals, and this was most strongly evident for right ear PPI under visuospatial attention manipulation. Meditators performed better than meditation-naïve individuals on attention tasks (verbal: more targets detected; visuospatial: faster reaction time). Spatial attention processes contribute to monaural PPI, particularly with the right ear. Better attentional performance, with similar attentional modulation of PPI, may indicate a stronger attentional capacity in meditators, relative to meditation-naïve individuals.