Andrew Bell

The impossibility of separating age, period and cohort effects

This commentary discusses the age-period-cohort identification problem. It shows that, despite a plethora of proposed solutions in the literature, no model is able to solve the identification problem because the identification problem is inherent to the real-world processes being modelled. As such, we cast doubt on the conclusions of a number of papers, including one presented here (Page, Milner, Morrell, & Taylor, 2013). We conclude with some recommendations for those wanting to model age, period and cohort in a compelling way.

Don't birth cohorts matter? A commentary and simulation exercise on Reither, Hauser and Yang's (2009) age-period-cohort study of obesity

Reither, Hauser, and Yang (2009) use a Hierarchical Age-Period-Cohort model (HAPC - Yang & Land, 2006) to assess changes in obesity in the USA population. Their results suggest that there is only a minimal effect of cohorts, and that it is periods which have driven the increase in obesity over time. We use simulations to show that this result may be incorrect. Using simulated data in which it is cohorts, rather than periods, that are responsible for the rise in obesity, we are able to replicate the period-trending results of Reither etxa0al. In this instance, the HAPC model misses the true cohort trend entirely, erroneously finds a period trend, and underestimates the age trend. Reither etxa0al.s results may be correct, but because age, period and cohort are confounded there is no way to tell. This is typical of age-period-cohort models, and shows the importance of caution when any APC model is used. We finish with a discussion of ways forward for researchers wishing to model age, period and cohort in a robust and non-arbitrary manner.

Life-course and cohort trajectories of mental health in the UK, 1991-2008: A multilevel age-period-cohort analysis

There is ongoing debate regarding the shape of life-course trajectories in mental health. Many argue the relationship is U-shaped, with mental health declining with age to mid-life, then improving. However, I argue that these models are beset by the age-period-cohort (APC) identification problem, whereby age, cohort and year of measurement are exactly collinear and their effects cannot be meaningfully separated. This means an apparent life-course effect could be explained by cohorts. This paper critiques two sets of literature: the substantive literature regarding life-course trajectories in mental health, and the methodological literature that claims erroneously to have solved the APC identification problem statistically (e.g. using Yang and Lands Hierarchical APC-HAPC-model). I then use a variant of the HAPC model, making strong but justified assumptions that allow the modelling of life-course trajectories in mental health (measured by the General Health Questionnaire) net of any cohort effects, using data from the British Household Panel Survey, 1991-2008. The model additionally employs a complex multilevel structure that allows the relative importance of spatial (households, local authority districts) and temporal (periods, cohorts) levels to be assessed. Mental health is found to increase throughout the life-course; this slows at mid-life before worsening again into old age, but there is no evidence of a U-shape--I argue that such findings result from confounding with cohort processes (whereby more recent cohorts have generally worse mental health). Other covariates were also evaluated; income, smoking, education, social class, urbanity, ethnicity, gender and marriage were all related to mental health, with the latter two in particular affecting life-course and cohort trajectories. The paper shows the importance of understanding APC in life-course research generally, and mental health research in particular.

Should age-period-cohort analysts accept innovation without scrutiny? A response to Reither, Masters, Yang, Powers, Zheng, and Land

This commentary clarifies our original commentary (Bell and Jones, 2014c) and illustrates some concerns we have regarding the response article in this issue (Reither et al., 2015). In particular, we argue that (a) linear effects do not have to be produced by exact linear mathematical functions to behave as if they were linear, (b) linear effects by this wider definition are extremely common in real life social processes, and (c) in the presence of these effects, the Hierarchical Age Period Cohort (HAPC) model will often not work. Although Reither et al. do not define what a non-linear monotonic trend is (instead, only stating that it isnt a linear effect) we show that the model often doesnt work in the presence of such effects, by using data generated as a non-linear monotonic trend by Reither et al. themselves. We then question their discussion of fixed and random effects before finishing with a discussion of how we argue that theory should be used, in the context of the obesity epidemic.

Formula for success: Multilevel modelling of Formula One Driver and Constructor performance, 1950–2014

Abstract This paper uses random-coefficient models and (a) finds rankings of who are the best formula 1 (F1) drivers of all time, conditional on team performance; (b) quantifies how much teams and drivers matter; and (c) quantifies how team and driver effects vary over time and under different racing conditions. The points scored by drivers in a race (standardised across seasons and Normalised) is used as the response variable in a cross-classified multilevel model that partitions variance into team, team-year and driver levels. These effects are then allowed to vary by year, track type and weather conditions using complex variance functions. Juan Manuel Fangio is found to be the greatest driver of all time. Team effects are shown to be more important than driver effects (and increasingly so over time), although their importance may be reduced in wet weather and on street tracks. A sensitivity analysis was undertaken with various forms of the dependent variable; this did not lead to substantively different conclusions. We argue that the approach can be applied more widely across the social sciences, to examine individual and team performance under changing conditions.

Understanding and misunderstanding group mean centering: a commentary on Kelley et al.’s dangerous practice

Kelley et al. argue that group-mean-centering covariates in multilevel models is dangerous, since—they claim—it generates results that are biased and misleading. We argue instead that what is dangerous is Kelley et al.’s unjustified assault on a simple statistical procedure that is enormously helpful, if not vital, in analyses of multilevel data. Kelley et al.’s arguments appear to be based on a faulty algebraic operation, and on a simplistic argument that parameter estimates from models with mean-centered covariates must be wrong merely because they are different than those from models with uncentered covariates. They also fail to explain why researchers should dispense with mean-centering when it is central to the estimation of fixed effects models—a common alternative approach to the analysis of clustered data, albeit one increasingly incorporated within a random effects framework. Group-mean-centering is, in short, no more dangerous than any other statistical procedure, and should remain a normal part of multilevel data analyses where it can be judiciously employed to good effect.

Age, Period and Cohort Processes in Longitudinal and Life Course Analysis: A Multilevel Perspective

This chapter considers age, period and cohort (APC) as different sources of health-related change. Age (or, life course) effects are individual, often biological, sources of change, whilst periods and cohorts can be thought of as social contexts affecting individuals that reside within them. Due to the mathematical confounding of age, period and cohort, careful consideration of each is important – otherwise what appears to be, for example, a period (year) effect could in fact be a mixture of age and cohort processes. Naive life course approaches could thus produce misleading results when APC effects are not all considered. However, the mathematical confounding also often makes modelling all three effects together impossible, and the dangers of attempting to do so, or of ignoring one effect without critical forethought, is illustrated through the example of the obesity epidemic. This example uses Yang and Land’s Hierarchical APC model which it is claimed (incorrectly) solves the identification problem. Finally, we suggest a flexible multilevel framework that extends Yang and Land’s model, and by making relatively strong assumptions (in this case that there are no long-run period trends) can model age, period and cohort effects robustly and explicitly, so long as those assumptions are correct. This is illustrated using health data from the British Household Panel Survey. We argue that this theory driven approach is often the most appropriate for conceptualising APC effects, and producing valid empirical inference about both individual life courses and the spatial and temporal contexts in which they exist.

The hierarchical age–period–cohort model: Why does it find the results that it finds?

Abstract It is claimed the hierarchical-age–period–cohort (HAPC) model solves the age–period–cohort (APC) identification problem. However, this is debateable; simulations show situations where the model produces incorrect results, countered by proponents of the model arguing those simulations are not relevant to real-life scenarios. This paper moves beyond questioning whether the HAPC model works, to why it produces the results it does. We argue HAPC estimates are the result not of the distinctive substantive APC processes occurring in the dataset, but are primarily an artefact of the data structure—that is, the way the data has been collected. Were the data collected differently, the results produced would be different. This is illustrated both with simulations and real data, the latter by taking a variety of samples from the National Health Interview Survey (NHIS) data used by Reither et al. (Soc Sci Med 69(10):1439–1448, 2009) in their HAPC study of obesity. When a sample based on a small range of cohorts is taken, such that the period range is much greater than the cohort range, the results produced are very different to those produced when cohort groups span a much wider range than periods, as is structurally the case with repeated cross-sectional data. The paper also addresses the latest defence of the HAPC model by its proponents (Reither et al. in Soc Sci Med 145:125–128, 2015a). The results lend further support to the view that the HAPC model is not able to accurately discern APC effects, and should be used with caution when there appear to be period or cohort near-linear trends.

Cross-Classified Multilevel Modelling of the Effectiveness of Similarity-Based Virtual Screening

The screening effectiveness of a chemical similarity search depends on a range of factors, including the bioactivity of interest, the types of similarity coefficient and fingerprint that comprise the similarity measure, and the nature of the reference structure that is being searched against a database. This study introduces the use of cross‐classified multilevel modelling as a way to investigate the relative importance of these four factors when carrying out similarity searches on the ChEMBL database. Two principal conclusions can be drawn from the analyses: that the fingerprint plays a more important role than the similarity coefficient in determining the effectiveness of a similarity search, and that comparative studies of similarity measures should involve many more reference structures than has been the case in much previous work.