M Egan

Publication bias in qualitative research: what becomes of qualitative research presented at conferences?

Background: Less than half of studies presented at conferences remain unpublished two years later, and these studies differ systematically from those that are published. In particular, the unpublished studies are less likely to report statistically significant findings, and this introduces publication bias. This has been well documented for quantitative studies, but has never been explored in relation to qualitative research. Methods: We reviewed the abstracts of qualitative research presented at the 1998 (n = 110) and 1999 (n = 114) British Sociological Association (BSA) Medical Sociology meetings, and attempted to locate those studies in databases or by contacting authors. We also appraised the quality of reporting in each abstract. Results: We found an overall publication rate for these qualitative studies of 44.2%. This is nearly identical to the publication rate for quantitative research. The quality of reporting of study methods and findings in the abstract was positively related to the likelihood of publication. Conclusion: Qualitative research is as likely to remain unpublished as quantitative research. Moreover, non-publication appears to be related to the quality of reporting of methodological information in the original abstract, perhaps because this is a proxy for a study with clear objectives and clear findings. This suggests a mechanism by which “qualitative publication bias” might work: qualitative studies that do not show clear, or striking, or easily described findings may simply disappear from view. One implication of this is that, as with quantitative research, systematic reviews of qualitative studies may be biased if they rely only on published papers.

Physical and mental health outcomes following housing improvements: evidence from the GoWell study

Background Existing research points towards physical and mental health gains from housing improvements, but findings are inconsistent and often not statistically significant. The detailed characteristics and variability of housing improvement works are problematic and studies are often small, not experimental, with short follow-up times. Methods A quasi-experimental design was used to assess the impact on physical health and mental health (using SF-12v2 Physical and Mental health component summary scales) of four types of housing improvement works—central heating, ‘Secured By Design’ front doors, fabric works, kitchens and bathrooms—both singly and in pairwise combinations. A longitudinal sample of 1933 residents from 15 deprived communities in Glasgow, UK was constructed from surveys carried out in 2006, 2008 and 2011. Sociodemographic characteristics and changes in employment status were taken into account. Results Fabric works had positive associations with physical health (+2.09, 95% CI 0.13 to 4.04) and mental health (+1.84, 95% CI 0.04 to 3.65) in 1–2 years. Kitchens and bathrooms had a positive association with mental health in 1–2 years (+2.58, 95% CI 0.79 to 4.36). Central heating had a negative association with physical health (−2.21, 95% CI −3.74 to −0.68). New front doors had a positive association with mental health in <1 year (+5.89, 95% CI 0.65 to 11.14) and when provided alongside kitchens and bathrooms (+4.25, 95% CI 1.71 to 6.80). Gaining employment had strong associations with physical health (+7.14, 95% CI 4.72 to 9.55) as well as mental health (+5.50, 95% CI 3.27 to 7.73). Conclusions Fabric works may provide insulation benefits and visual amenity benefits to residents. Front doors may provide important security benefits in deprived communities. Economic regeneration is important alongside property-led regeneration.

Considering methodological options for reviews of theory: illustrated by a review of theories linking income and health

BackgroundReview of theory is an area of growing methodological advancement. Theoretical reviews are particularly useful where the literature is complex, multi-discipline, or contested. It has been suggested that adopting methods from systematic reviews may help address these challenges. However, the methodological approaches to reviews of theory, including the degree to which systematic review methods can be incorporated, have received little discussion in the literature. We recently employed systematic review methods in a review of theories about the causal relationship between income and health.MethodsThis article discusses some of the methodological issues we considered in developing the review and offers lessons learnt from our experiences. It examines the stages of a systematic review in relation to how they could be adapted for a review of theory. The issues arising and the approaches taken in the review of theories in income and health are considered, drawing on the approaches of other reviews of theory.ResultsDifferent approaches to searching were required, including electronic and manual searches, and electronic citation tracking to follow the development of theories. Determining inclusion criteria was an iterative process to ensure that inclusion criteria were specific enough to make the review practical and focused, but not so narrow that key literature was excluded. Involving subject specialists was valuable in the literature searches to ensure principal papers were identified and during the inductive approaches used in synthesis of theories to provide detailed understanding of how theories related to another. Reviews of theory are likely to involve iterations and inductive processes throughout, and some of the concepts and techniques that have been developed for qualitative evidence synthesis can be usefully translated to theoretical reviews of this kind.ConclusionsIt may be useful at the outset of a review of theory to consider whether the key aim of the review is to scope out theories relating to a particular issue; to conduct in-depth analysis of key theoretical works with the aim of developing new, overarching theories and interpretations; or to combine both these processes in the review. This can help decide the most appropriate methodological approach to take at particular stages of the review.

Health improvement programmes: really too complex to evaluate?

Imagine an intervention whose effects vary within and between individuals and depend on subtle interactions between deliverers and recipients, and in which exposure is uncertain. Given this complexity, who would contemplate conducting a randomised controlled trial? In fact, all these issues must be dealt with in drug or therapeutic trials, as well as in …

[Accepted Manuscript] Local policies to tackle a national problem: Comparative qualitative case studies of an English local authority alcohol availability intervention.

Cumulative impact policies (CIPs) are widely used in UK local government to help regulate alcohol markets in localities characterised by high density of outlets and high rates of alcohol related harms. CIPs have been advocated as a means of protecting health by controlling or limiting alcohol availability. We use a comparative qualitative case study approach (n=5 English local government authorities, 48 participants) to assess how CIPs vary across different localities, what they are intended to achieve, and the implications for local-level alcohol availability. We found that the case study CIPs varied greatly in terms of aims, health focus and scale of implementation. However, they shared some common functions around influencing the types and managerial practices of alcohol outlets in specific neighbourhoods without reducing outlet density. The assumption that this will lead to alcohol harm-reduction needs to be quantitatively tested.

You Can't Always Get What You Want...? Prior-Attitudes and Post-Experiences of Relocation from Restructured Neighbourhoods

This study uses a longitudinal, qualitative research methodology to compare residents’ prior attitudes towards relocation from restructured neighbourhoods with their experiences post-move. Participants were householders in families with children, with interviews carried out shortly before, and up to 18 months after relocation. There was generally a good fit between prior attitudes and post-experiences, although those who had not wanted to move reported more gains than expected, and those who had wanted to move to ‘get on’ with their lives had yet to make major changes in their lives after relocation. There was some retrospective reassessment of prior attitudes after relocation, consistent with the notion of low expectations among deprived area residents. There were both social and psychosocial gains from relocation, with a weak prior sense of community and inconsistent effects of distance upon social outcomes. Important mediators of adult experiences and outcomes were personality, health status and relations with children.

P66 Cross-sectional and longitudinal associations between alcohol licensing policies, outlet density and deprivation and population health and crime in England

Background An important policy area to address the cumulative effects of the use of alcohol on communities is that of regulating the physical availability of alcohol and modifying the commercial drinking environment. Such policies may not be directly aimed at public health, but primarily aim to target crime and anti-social behaviour, and can have a downstream effect on population health as well. Here, we analysed the effect of the intensity of local area alcohol licensing policy and enforcement on the cumulative impact on crimes and population health. Methods Home Office Alcohol and Late Night Refreshment Licensing data was linked to local-area alcohol-related hospital admission from the Local Alcohol Profiles for England, population size and deprivation. Cumulative policy and enforcement intensity (hereforth ‘intensity’) was coded as ‘passive’, medium or high based on presence of cumulative impact zones and/or successfully challenged licenses applications. Temporal trends in directly age-standardised rates alcohol-related hospital admissions and rates of reported (alcohol-related) violent crimes, sex crimes, public order offences, and other crimes for 2009–15 were analysed using mixed-effects log-rate models adjusted for seasonality, population size, deprivation, and alcohol outlet density. Results Crimes and alcohol-related hospital admissions were moderately to strongly correlated at the local level, illustrating the potential cumulative burden on certain areas partly attributable to alcohol use. Cross-sectionally, alcohol-related population harms and alcohol outlet density were higher in more deprived areas, but these were also the areas with the highest policy and enforcement ‘intensity’; indicating that introduction of licensing policies was not done randomly but targets specific, and on average the correct, areas. Longitudinally, ‘exposure’-response associations were observed, with an additional average decrease in alcohol-related hospital admission rates in the areas with the highest ‘intensity’ of 2% (95% CI: −3 to −2%) annually (P = 0.006). Patterns for crime rates were more complex, but indicate that in 2009–2012 stronger reductions were similarly observed in more ‘intense areas’. This pattern was not observed for ‘financial fraud’, which we characterised as not related to alcohol consumption. Post-2012 however, crime rates increased again, but faster in more ‘intense’ areas (although changes in reporting requires cautious interpretation). Conclusion These results add to the available evidence about the potential benefits of alcohol licensing policies and enforcement in England. Moreover, they indicate that alcohol licensing policies are part of a wider policy package aimed at tackling cumulative burdens of more general crimes and anti-social behaviour and which positively affects population health.

PP42 A mixed methods evaluation of a local-level alcohol availability intervention: “reducing the strength”

Background Reducing the local availability of alcohol may reduce alcohol-related harms. In the UK, local authorities are becoming increasingly interested in an intervention called ‘Reducing the Strength’ (RtS) whereby off-licence shops and supermarkets voluntarily stop selling inexpensive super strength (≥6.5% alcohol by volume) beers and ciders. We conducted a mixed methods evaluation to describe, measure and understand the effects of this example of local policy innovation. Methods The qualitative phase involved documentary analysis and interviews with public health, police, licensing and retail professionals (n = 15) to gain multi-sectoral perspectives of the intervention, its aims, implementation and perceived impacts. The quantitative phase involved difference-in-differences analysis of data from a large retail chain: data on unit alcohol sales were obtained for three UK counties (131 stores). In one county the intervention started 12 months earlier than the others, allowing for a pre–post study design with a delayed implementation comparator. Results Qualitative findings highlighted multiple perspectives from different stakeholders. Those responsible for intervention development emphasised the need to link it with support services and policing strategies. Street drinkers were the initial target for the intervention, but public health practitioners suggested mechanisms for broader population impacts through reduced purchasing and consumption of alcohol units. Our retail analysis found that during the baseline/pre-intervention period, the intervention and comparison areas sold, on average, 170393 and 183296 alcohol units per store (respectively), and 174703 and 201453 units at follow up. Therefore both intervention and comparison areas experienced increased sales at follow-up, with the difference between the two areas widening from 12903 to 26750 (p = 0.690). Conclusion Findings from our quantitative analysis were inconclusive. The implementation did demonstrate the feasibility of co-opting retailer and public sectors into strategies that linked community safety, store security and public health goals.

P1-186 Challenges in natural experiment research: potential lessons from the H1N1 pandemic

Introduction Research on natural experiments (NEs) allows investigation into topics that are important for policy-makers, including many social determinants of health. NEs can be conceptualised as differing from other research in three important ways. First, unlike trials, investigators cannot allocate the exposure of interest which is instead externally decided. Second, researchers are unable to influence the exposures characteristics which results in uncertainty of the fidelity, dose and whether the NE will occur at all. Third, and in contrast to many observational studies, the timing of the exposure and hence research itself, cannot be chosen by researchers. Methods Using three case studies of research on H1N1 (two quantitative sero-epidemiology studies, one qualitative study) we identified key factors that allowed us to successfully conduct our investigation of a NE. We compared these factors with the wider NE literature to identify common barriers and facilitators to research. Results We identified key external factors that influence capacity to successfully take advantage of NEs. As a result of the uncertainty of a NE occurring and the rapid response required, flexibility is needed by researchers, policymakers, practitioners, ethics committees and funders. This is achievable when research is perceived as immediately important for health (as in H1N1) but may otherwise be difficult. Discussion We suggest researchers investigating NEs face additional challenges to traditional observational studies. While some barriers are insurmountable, actions such as rapid response funding, fast-track ethics procedures and improvements in routine data can create a more conducive environment allowing policy-relevant evaluation.

P1-402 Psychosocial pathways to mental well-being in deprived areas

Introduction Housing-led regeneration has been shown to have limited effects on mental health. Considering neighbourhoods as a psychosocial environment, regeneration may have greater impact on mental well-being than mental ill-health. This study examined the relationship between the well-being of residents living in deprived areas and aspects of housing, neighbourhoods and communities. Methods A cross-sectional study of 3911 residents in 15 deprived areas in Glasgow, Scotland. Mental well-being was measured using the Warwick Edinburgh Mental Well-being Scale. Results Using multivariate analysis and controlling for socio-demographic characteristics and physical health status, we found that residential and environmental aspects of peoples houses and neighbourhoods were strongly associated with well-being: when respondents considered that their neighbourhood had very good aesthetic qualities (RR 3.3, 95% CI 1.9 to 5.8), their home and neighbourhood represented personal progress (RR 3.2 95% CI 2.2 to 4.8; RR 2.6, 95% CI 1.8 to 3.7, respectively), they perceived their residence to have a very good external appearance (RR 2.6, 95% CI 1.3 to 5.1); a very good front door (both an aesthetic and a security/control item) (RR 2.1, 95% CI 1.2 to 3.8), and satisfaction with their landlord was very high (RR 2.3, 95% CI 2.2 to 4.8). Perception of poor neighbourhood aesthetic quality was associated with lower well-being (RR 0.4, 95% CI 0.3 to 0.5). Discussion This study has found that where we live matters for mental well-being. In particular, it appears that positive residential circumstances may influence how we feel about ourselves and our view of our position in society, with beneficial consequences for well-being.