Alexander Krauss

Why all randomised controlled trials produce biased results

Abstract Background: Randomised controlled trials (RCTs) are commonly viewed as the best research method to inform public health and social policy. Usually they are thought of as providing the most rigorous evidence of a treatment’s effectiveness without strong assumptions, biases and limitations. Objective: This is the first study to examine that hypothesis by assessing the 10 most cited RCT studies worldwide. Data sources: These 10 RCT studies with the highest number of citations in any journal were identified by searching Scopus (the largest database of peer-reviewed journals). Results: This study shows that these world-leading RCTs that have influenced policy produce biased results by illustrating that participants’ background traits that affect outcomes are often poorly distributed between trial groups, that the trials often neglect alternative factors contributing to their main reported outcome and, among many other issues, that the trials are often only partially blinded or unblinded. The study here also identifies a number of novel and important assumptions, biases and limitations not yet thoroughly discussed in existing studies that arise when designing, implementing and analysing trials. Conclusions: Researchers and policymakers need to become better aware of the broader set of assumptions, biases and limitations in trials. Journals need to also begin requiring researchers to outline them in their studies. We need to furthermore better use RCTs together with other research methods. Key messages RCTs face a range of strong assumptions, biases and limitations that have not yet all been thoroughly discussed in the literature. This study assesses the 10 most cited RCTs worldwide and it shows, more generally, that trials inevitably produce bias. Trials involve complex processes – from randomising, blinding and controlling, to implementing treatments, monitoring participants etc. – that require many decisions and steps at different levels that bring their own assumptions and degree of bias to results.

Subjective Wellbeing in Colombia: Some Insights on Vulnerability, Job Security, and Relative Incomes

A burgeoning literature explores the extent to which consumption or income inadequately reflect peoples subjective wellbeing, just as GDP at times can provide an incomplete and misleading picture of national wellbeing. Scholars are increasingly using data on subjective wellbeing to complement traditional welfare indicators and to enrich our understanding of wellbeing and quality of life. The paper builds on the present research but it analyzes a much broader, more interdisciplinary, and more policy-relevant range of potential determinants simultaneously than currently existing in the literature on subjective wellbeing. It first analyzes the relative importance of a wide range of characteristics and conditions at the individual, household, regional and macro levels on levels of subjective wellbeing in Colombia in 2010/11; and second, assesses the marginal effects of a number of factors on perceived changes in levels of subjective wellbeing over time for the same respondents from 2008/09 to 2010/11. Findings show that increasing the quality of life of Colombians is largely conditional on minimizing risks and vulnerabilities: reducing the rate and duration of unemployment; improving the delivery of public health services; increasing the share of people with health and pension plans; enhancing safety and security in communities; and reducing levels of discrimination. It finds that job loss has particularly strong effects on levels of satisfaction that are larger than those for increased income, while also controlling for a decrease in income that is often related to being unemployed, suggesting that the human welfare (non-pecuniary) costs of unemployment are driving the strong effects. Moreover, any job, even a low-quality job, is overall better for ones subjective wellbeing than being unemployed. Finally, policy aimed at improving peoples subjective wellbeing will likely have the greatest impact if focused on mitigating vulnerabilities and negative shocks that people face.

Understanding child labor beyond poverty: the structure of the economy, social norms, and no returns to rural basic education

Child labor is pervasive across sub-Saharan Africa. The common assumption is that monetary poverty is its most important cause. This paper investigates this hypothesis with empirical evidence by exploring structural, geographic, monetary, demographic, cultural, seasonal and school-supply factors simultaneously that can influence child labor. It is a first attempt in the literature to combine quantitative with qualitative methods to identify a broader range of potential factors—on the demand- and supply-side and at the micro and macro levels—for why children work in agrarian economies like Ghana. Interviews with the Minister of Education and with children enrich the multivariate regression results. The multiple sources of child labor appear to include, in particular, the structure of the economy, social norms and no returns to rural basic education. Policy responses are outlined especially on the demand side that are needed to help reduce harmful child labor that affects childrens education and later opportunities.