Silvia Rizzi

Efficient Estimation of Smooth Distributions From Coarsely Grouped Data

Ungrouping binned data can be desirable for many reasons: Bins can be too coarse to allow for accurate analysis; comparisons can be hindered when different grouping approaches are used in different histograms; and the last interval is often wide and open-ended and, thus, covers a lot of information in the tail area. Age group–specific disease incidence rates and abridged life tables are examples of binned data. We propose a versatile method for ungrouping histograms that assumes that only the underlying distribution is smooth. Because of this modest assumption, the approach is suitable for most applications. The method is based on the composite link model, with a penalty added to ensure the smoothness of the target distribution. Estimates are obtained by maximizing a penalized likelihood. This maximization is performed efficiently by a version of the iteratively reweighted least-squares algorithm. Optimal values of the smoothing parameter are chosen by minimizing Akaikes Information Criterion. We demonstrate the performance of this method in a simulation study and provide several examples that illustrate the approach. Wide, open-ended intervals can be handled properly. The method can be extended to the estimation of rates when both the event counts and the exposures to risk are grouped.

Comparison of non-parametric methods for ungrouping coarsely aggregated data

BackgroundHistograms are a common tool to estimate densities non-parametrically. They are extensively encountered in health sciences to summarize data in a compact format. Examples are age-specific distributions of death or onset of diseases grouped in 5-years age classes with an open-ended age group at the highest ages. When histogram intervals are too coarse, information is lost and comparison between histograms with different boundaries is arduous. In these cases it is useful to estimate detailed distributions from grouped data.MethodsFrom an extensive literature search we identify five methods for ungrouping count data. We compare the performance of two spline interpolation methods, two kernel density estimators and a penalized composite link model first via a simulation study and then with empirical data obtained from the NORDCAN Database. All methods analyzed can be used to estimate differently shaped distributions; can handle unequal interval length; and allow stretches of 0 counts.ResultsThe methods show similar performance when the grouping scheme is relatively narrow, i.e. 5-years age classes. With coarser age intervals, i.e. in the presence of open-ended age groups, the penalized composite link model performs the best.ConclusionWe give an overview and test different methods to estimate detailed distributions from grouped count data. Health researchers can benefit from these versatile methods, which are ready for use in the statistical software R. We recommend using the penalized composite link model when data are grouped in wide age classes.

Why did Danish women's life expectancy stagnate? The influence of interwar generations' smoking behaviour.

The general health status of a population changes over time, generally in a positive direction. Some generations experience more unfavourable conditions than others. The health of Danish women in the interwar generations is an example of such a phenomenon. The stagnation in their life expectancy between 1977 and 1995 is thought to be related to their smoking behaviour. So far, no study has measured the absolute effect of smoking on the mortality of the interwar generations of Danish women and thus the stagnation in Danish women’s life expectancy. We applied a method to estimate age-specific smoking-attributable number of deaths to examine the effect of smoking on the trends in partial life expectancy of Danish women between age 50 and 85 from 1950 to 2012. We compared these trends to those for women in Sweden, where there was no similar stagnation in life expectancy. When smoking-attributable mortality was excluded, the gap in partial life expectancy at age 50 between Swedish and Danish women diminished substantially. The effect was most pronounced in the interwar generations. The major reason for the stagnation in Danish women’s partial life expectancy at age 50 was found to be smoking-related mortality in the interwar generations.

ungroup: An R package for efficient estimation of smooth distributions from coarsely binned data

ungroup is an open source software library written in the R programming language (R Core Team, 2018) that introduces a versatile method for ungrouping histograms (binned count data) assuming that counts are Poisson distributed and that the underlying sequence over a fine grid to be estimated is smooth. The method is based on the composite link model (Thompson & Baker, 1981) and estimation is achieved by maximizing a penalized likelihood (P. H. Eilers, 2007), which extends standard generalized linear models. The penalized composite link model (PCLM) implements the idea that observed counts, interpreted as realizations from Poisson distributions, are indirect observations of a finer (ungrouped) but latent sequence. This latent sequence represents the distribution of expected means on a fine resolution and has to be estimated from the aggregated data. Estimates are obtained by maximizing a penalized likelihood. This maximization is performed efficiently by a version of the iteratively re-weighted least-squares algorithm. Optimal values of the smoothing parameter are chosen by minimizing Bayesian or Akaike’s Information Criterion (Hastie & Tibshirani, 1990).

How to estimate mortality trends from grouped vital statistics

Abstract Background Mortality data at the population level are often aggregated in age classes, for example 5-year age groups with an open-ended interval for the elderly aged 85+. Capturing detailed age-specific mortality patterns and mortality time trends from such coarsely grouped data can be problematic at older ages, especially where open-ended intervals are used. Methods We illustrate the penalized composite link model (PCLM) for ungrouping to model cancer mortality surfaces. Smooth age-specific distributions from data grouped in age classes of adjacent calendar years were estimated by constructing a two-dimensional regression, based on B-splines, and maximizing a penalized likelihood. We show the applicability of the proposed model, analysing age-at-death distributions from cancers of all sites in Denmark from 1980 to 2014. Data were retrieved from the Danish Cancer Society and the Human Mortality Database. Results The main trends captured by PCLM are: (i) a decrease in cancer mortality rates after the 1990s for ages 50–75; (ii) a decrease in cancer mortality in later cohorts for young ages, especially, and very advanced ages. Comparing the raw data by single year of age, with the PCLM-ungrouped distributions, we clearly illustrate that the model fits the data with a high level of accuracy. Conclusions The PCLM produces detailed smooth mortality surfaces from death counts observed in coarse age groups with modest assumptions, that is Poisson distributed counts and smoothness of the estimated distribution. Hence, the method has great potential for use within epidemiological research when information is to be gained from aggregated data, because it avoids strict assumptions about the actual distributional shape.

Comparison of cognitive and physical functioning of Europeans in 2004-05 and 2013

Abstract Background Adult mortality has been postponed over time to increasingly high ages. However, evidence on past and current health trends has been mixed, and little is known about European disability trends. Methods In a cross-sectional setting, we compared cognitive and physical functioning in same-aged Europeans aged 50+ between 2004–05 (wave 1; n = 18 757) and 2013 (wave 5 refresher respondents; n = 16 696), sourced from the Survey of Health, Ageing and Retirement in Europe (SHARE). Results People in 2013 had better cognitive function compared with same-aged persons in 2004–05, with an average difference of approximately one-third standard deviation. The same level of cognitive function in 2004–05 at age 50 was found in 2013 for people who were 8 years older. There was an improvement in cognitive function in all European regions. Mean grip strength showed an improvement in Northern Europe of 1.00 kg [95% confidence interval (CI) 0.65; 1.35] and in Southern Europe of 1.68 kg (95% CI 1.14; 2.22), whereas a decrease was found in Central Europe (-0.80 kg; 95% CI −1.16; −0.44). We found no overall differences in activities of daily living (ADL), but small improvements in instrumental activities of daily living (IADL) in Northern and Southern Europe, with an improvement in both ADL and IADL from age 70 in Northern Europe. Conclusions Our results indicate that later-born Europeans have substantially better cognitive functioning than earlier-born cohorts. For physical functioning, improvements were less clear, but for Northern Europe there was an improvement in ADL and IADL in the oldest age groups.