David G Steel

Investigation of relative risk estimates from studies of the same population with contrasting response rates and designs

BackgroundThere is little empirical evidence regarding the generalisability of relative risk estimates from studies which have relatively low response rates or are of limited representativeness. The aim of this study was to investigate variation in exposure-outcome relationships in studies of the same population with different response rates and designs by comparing estimates from the 45 and Up Study, a population-based cohort study (self-administered postal questionnaire, response rate 18%), and the New South Wales Population Health Survey (PHS) (computer-assisted telephone interview, response rate ~60%).MethodsLogistic regression analysis of questionnaire data from 45 and Up Study participants (n = 101,812) and 2006/2007 PHS participants (n = 14,796) was used to calculate prevalence estimates and odds ratios (ORs) for comparable variables, adjusting for age, sex and remoteness. ORs were compared using Wald tests modelling each study separately, with and without sampling weights.ResultsPrevalence of some outcomes (smoking, private health insurance, diabetes, hypertension, asthma) varied between the two studies. For highly comparable questionnaire items, exposure-outcome relationship patterns were almost identical between the studies and ORs for eight of the ten relationships examined did not differ significantly. For questionnaire items that were only moderately comparable, the nature of the observed relationships did not differ materially between the two studies, although many ORs differed significantly.ConclusionsThese findings show that for a broad range of risk factors, two studies of the same population with varying response rate, sampling frame and mode of questionnaire administration yielded consistent estimates of exposure-outcome relationships. However, ORs varied between the studies where they did not use identical questionnaire items.

Using census data to investigate the causes of the ecological fallacy

The authors show how data from the 2% Sample of Anonymised Records (SAR) can be combined with data from the Small Area Statistics (SAS) database to investigate the causes of the ecological fallacy in an Enumeration District (ED) level analysis. A range of census variables are examined in three ‘SAR districts’ (local authority districts with populations of 120 000 or more, or combinations of contiguous districts with smaller populations) in England. Results of comparable analyses from the 1986 Australian census are also given. The ecological fallacy arises when results from an analysis based on area-level aggregate statistics are incorrectly assumed to apply at the individual level. In general the results are different because individuals in the same area tend to have similar characteristics: a phenomenon known as within-area homogeneity. A statistical model is presented which allows for within-area homogeneity. This model may be used to explain the effects of aggregation on variances, covariances, and correlations. A methodology is introduced which allows aggregate-level statistics to be adjusted by using individual-level information on those variables that explain much of the within-area homogeneity. This methodology appears to be effective in adjusting census data analyses, and the results suggest that the SAR is a valuable source of adjustment information for aggregate data analyses from census and other sources.

Scales, levels and processes: Studying spatial patterns of British census variables

Abstract This paper is based on the assumption that there may be scale effects at all levels of areal data and that they vary both within areal units and between areal units. Spatial distributions are based on processes taking place in geographical space. A mapped pattern may reflect several distinct processes, each of which may affect a different area and operate at a different scale. The challenge for the spatial analyst is to identify these processes and evaluate their importance from the spatial pattern observed. Here the well known modifiable areal unit problem is not really a problem but a resource. Data at different scales can help us identify processes operating at different scales. We build on models and methods described by [Tranmer, M., & Steel, D. G. (2001). Using local census data to investigate scale effects. In N. J. Tate, & P. M. Atkinson (Eds.), Modelling scale in geographical information science (pp. 105–122). Chichester: John Wiley and Sons], which facilitate the identification of processes occurring within areal units. The method is extended using concepts from multi-level modelling and spatial autocorrelation, through the application of local statistics applied to what may be termed area effect estimates. It is illustrated with respect to two very different census variables and three different study areas.

Cohort profile: The Dynamic Analyses to Optimize Ageing (DYNOPTA) project

National Health and Medical Research Council (410215); NHMRC Fellowships (#366756 to K.J.A. and #316970 to P.B.)

Estimates of probable dementia prevalence from population-based surveys compared with dementia prevalence estimates based on meta-analyses

BackgroundNational data on dementia prevalence are not always available, yet it may be possible to obtain estimates from large surveys that include dementia screening instruments. In Australia, many of the dementia prevalence estimates are based on European data collected between 15 and 50 years ago. We derived population-based estimates of probable dementia and possible cognitive impairment in Australian studies using the Mini-Mental State Examination (MMSE), and compared these to estimates of dementia prevalence from meta-analyses of European studies.MethodsData sources included a pooled dataset of Australian longitudinal studies (DYNOPTA), and two Australian Bureau of Statistics National Surveys of Mental Health and Wellbeing. National rates of probable dementia (MMSE < 24) and possible cognitive impairment (24-26) were estimated using combined sample weights.ResultsEstimates of probable dementia were higher in surveys than in meta-analyses for ages 65-84, but were similar at ages 85 and older. Surveys used weights to account for sample bias, but no adjustments were made in meta-analyses. Results from DYNOPTA and meta-analyses had a very similar pattern of increase with age. Contrary to trends from some meta-analyses, rates of probable dementia were not higher among women in the Australian surveys. Lower education was associated with higher prevalence of probable dementia. Data from investigator-led longitudinal studies designed to assess cognitive decline appeared more reliable than government health surveys.ConclusionsThis study shows that estimates of probable dementia based on MMSE in studies where cognitive decline and dementia are a focus, are a useful adjunct to clinical studies of dementia prevalence. Such information and may be used to inform projections of dementia prevalence and the concomitant burden of disease.

Ignoring a level in a multilevel model: Evidence from UK census data

Because of the inherent multilevel nature of census data, it is often appropriate to use multilevel models to investigate relationships between census variables. For a local population, the data available from the census allow a three-level nested model to be assumed, with an individual level (level 1), an enumeration district (ED) level (level 2), and a ward level (level 3). The consequences of ignoring one of the three levels in this model are assessed here theoretically. Empirical results, based on 1991 UK Census data, are also provided, comparing the variance components estimated from the three-level model with analyses based on models where the ED or ward level are ignored. The results show how the variation that occurs at the level not included in the models is redistributed to the other levels that the models do include.

Rules for random aggregation

In this paper we derive the effect of aggregation on common statistics when the geographic areas used in the analysis are equivalent to randomly formed groups of individuals. Simple rules of aggregation are provided for use when analysis of such groups is performed. The expectations of common statistics such as means, variances, regression and correlation coefficients, are not affected by aggregation. However, the variation of these statistics is affected, mainly as a result of changes in the number of groups. This variation is related solely to random fluctuations associated with the generation of variate values. Weighting by the group population sizes is shown to be important in the calculation of statistics. Generally, unweighted statistics have larger variation than the corresponding weighted version, and the variation depends not only on the number of groups but also on the distribution of group population sizes. Methods for conducting statistical analysis of aggregate data in this situation are described and statistical inferences based on unweighted statistics are shown to be invalid.

The information in aggregate data

Ecological analysis involves using aggregate data for a set of groups to make inferences concerning individual level relationships. Typically the data available for analysis consists of the means or totals of variables of interest for geographical areas, although the groups can be organisations such as schools or hospitals. Attention has focused on developing methods of estimating the parameters characterising the individual level relationships across the whole population, but also in some cases the relationships for each of the groups. Applying standard methods used to analyse individual level data, such as linear or logistic regression or contingency table analysis, to aggregate data will usually produce biased estimates of individual level relationships. Thus much of the effort in ecological analysis has concentrated on developing methods of analysing aggregate data that can produce unbiased, or less biased, parameter estimates. There has been less work done on inference procedures, such as constructing confidence intervals and hypothesis testing. Fundamental to these inferential issues is the question of how much information is contained in aggregate data and what evidence such data can provide concerning important assumptions and hypotheses.

Simple methods for ecological inference in 2×2 tables

This paper considers inference about the individual level relationship between two dichotomous variables based on aggregated data. It is known that such analyses suffer from ‘ecological bias’, caused by the lack of homogeneity of this relationship across the groups over which the aggregation occurs. Two new methods for overcoming this bias, one based on local smoothing and the other a simple semiparametric approach, are developed and evaluated. The local smoothing approach performs best when it is used with a covariate which accounts for some of the variation in the relationships across groups. The semiparametric approach performed well in our evaluation even without such auxiliary information

Inclusion of mobile phone numbers into an ongoing population health survey in New South Wales, Australia: design, methods, call outcomes, costs and sample representativeness

BackgroundIn Australia telephone surveys have been the method of choice for ongoing jurisdictional population health surveys. Although it was estimated in 2011 that nearly 20% of the Australian population were mobile-only phone users, the inclusion of mobile phone numbers into these existing landline population health surveys has not occurred. This paper describes the methods used for the inclusion of mobile phone numbers into an existing ongoing landline random digit dialling (RDD) health survey in an Australian state, the New South Wales Population Health Survey (NSWPHS). This paper also compares the call outcomes, costs and the representativeness of the resultant sample to that of the previous landline sample.MethodsAfter examining several mobile phone pilot studies conducted in Australia and possible sample designs (screening dual-frame and overlapping dual-frame), mobile phone numbers were included into the NSWPHS using an overlapping dual-frame design. Data collection was consistent, where possible, with the previous years’ landline RDD phone surveys and between frames. Survey operational data for the frames were compared and combined. Demographic information from the interview data for mobile-only phone users, both, and total were compared to the landline frame using χ2 tests. Demographic information for each frame, landline and the mobile-only (equivalent to a screening dual frame design), and the frames combined (with appropriate overlap adjustment) were compared to the NSW demographic profile from the 2011 census using χ2 tests.ResultsIn the first quarter of 2012, 3395 interviews were completed with 2171 respondents (63.9%) from the landline frame (17.6% landline only) and 1224 (36.1%) from the mobile frame (25.8% mobile only). Overall combined response, contact and cooperation rates were 33.1%, 65.1% and 72.2% respectively. As expected from previous research, the demographic profile of the mobile-only phone respondents differed most (more that were young, males, Aboriginal and Torres Strait Islanders, overseas born and single) compared to the landline frame responders. The profile of respondents from the two frames combined, with overlap adjustment, was most similar to the latest New South Wales (NSW) population profile.ConclusionsThe inclusion of the mobile phone numbers, through an overlapping dual-frame design, did not impact negatively on response rates or data collection, and although costing more the design was still cost-effective because of the additional interviews that were conducted with young people, Aboriginal and Torres Strait Islanders and people who were born overseas resulting in a more representative overall sample.