Wim Kalmijn

From Discrete 1 to 10 towards Continuous 0 to 10: The Continuum Approach to Estimating the Distribution of Happiness in a Nation.

Happiness is often measured in surveys using responses to a single question with a limited number of response options, such as ‘very happy’, ‘fairly happy’ and ‘not too happy’. There is much variety in the wording and number of response options used, which limits comparability across surveys. To solve this problem, descriptive statistics of the discrete distribution in the sample are often transformed to a common discrete secondary scale, mostly ranging from 0 to 10. In an earlier publication we proposed a method for estimating statistics of the corresponding continuous distribution in the population (Kalmijn 2010). In the present paper we extend this method to questions using numerical response scales. The application of this ‘continuum approach’ to results obtained using the often used 1–10 numerical scale can make these comparable to those obtained on the basis of verbal response scales.

Conversion of Verbal Response Scales: Robustness Across Demographic Categories

Happiness and life satisfaction have traditionally been measured using verbal response scales, however, these verbal scales have not kept up with the present trend to use numerical response scales. A switch from a verbal scale to a numerical scale, however, causes a severe problem for trend analyses, due to the incomparability of the old and new measurements. The Reference Distribution Method is a method that has been developed recently to deal with this comparison problem. In this method use is made of a reference distribution based on responses to a numerical scale which is used to decide at which point verbally labelled response options transit from one state to another, for example from ‘happy’ to ‘very happy’. Next, for each wave of the time series in which the verbal scale is used, a population mean is estimated for the beta distribution that fits best to these transition points and the responses in this wave. These estimates are on a level that is comparable to that of the mean of the reference distribution and are appropriate for use in an extended time series based on the responses measured using a verbal and a numerical scale. In this paper we address the question of whether the transition points derived for the general population can be used for demographic categories to produce reliable, extended time series to monitor differences in trends among these categories. We conclude that this is possible and that it is not necessary to derive transition points for each demographic category separately.

The Reference Distribution Method

With the Reference Distribution Method an attempt is made to deal with the fact that, for a given year and a given population, one would expect the distribution means after scale transformation for similar questions about a certain topic asked in different representative surveys to be approximately the same irrespective of the primary response scales used. This provides the basis for another approach to transformation. The Reference Distribution Method for making survey data comparable builds heavily on the Scale Interval Method. Basically the two methods are identical except that in the Reference Distribution Method the boundaries between the response options of the primary scale are derived from a reference distribution instead of being derived from the assessments by judges by means of the Scale Interval Recorder.

Stability of the Boundaries Between Response Options for Different Countries

When the Continuum Approach is applied to a time series of a survey which has remained unchanged over time, the boundaries between the response options are assumed to remain unchanged over time. We inspect whether this assumption is realistic using time series of survey items taken from the Eurobarometer for different parts of Europe and on the topics life satisfaction and satisfaction with the way the democracy works in the country. We conclude that the research question can be answered affirmatively.

Combining and Pooling of Time Series on Life Satisfaction in the USA, Japan, The Netherlands and Spain

We applied the Reference Distribution Method to pool time series on life satisfaction in the USA, Japan, The Netherlands and Spain, using results from the World Values Survey to derive reference distributions from. This resulted in consistent time series spanning almost 60 years for Japan, 40 years for The Netherlands and 35 years for the USA and Spain. Life satisfaction in Japan and The Netherlands was almost equal in the eighties, but at present differs more than one point in favor of The Netherlands. Life satisfaction in Spain reached its lowest value of 6.0 in 2012, but has increased since then to 6.3 in 2015.

Diversity in Survey Items and the Comparability Problem

This chapter starts with an introduction to the incomparability problem and an overview of the diversity in survey items. This is followed by a description of the problem of incomparability of the time series on life satisfaction in the USA, Japan and The Netherlands to illustrate the problem. Next two conventional methods for scale transformation are described: the Linear Stretch Method and the Semantic Judgment of Fixed Word Value Method. We explain why they fall short to solve the comparability problem and conclude that these shortcomings require further investigations and innovative solutions to solve them.

The Labeling of Anchor Points and the Occurrence of Zero-Width Intervals

The labeling of the anchor points of a response scale may tempt the judges participating in scale intervals studies to assign zero-width intervals to these points by choosing the upper and lower bound equal to an extreme of the continuum. The phenomenon of assigning zero-width intervals occurs more frequently when extreme wording is used for an anchor point label and when the response scale consists of more than five response options. Although all-inclusive response scales would encompass the full spectrum of possible experiences, there uses is likely to make anchor points redundant for part of the respondents using such a response scale.

The Continuum Approach

The comparability problem is partly due to the variety of response scales that has developed over time in the field of survey research and is caused by the use of discrete scales. When conventional scale transformation methods are used or the Scale Interval Method to transform a discrete primary scale, the resulting secondary scale is still discrete, and although the use of discrete scales in survey research is often practically motivated, a more valid approach is to consider the existence of a latent continuous variable underlying the survey variable, the distribution of which is estimated using the survey item and the response to it. The Continuum Approach was developed with the notion that happiness should be treated as a continuous variable. In the Continuum Approach the shape parameters α and β of the best fit beta distribution are estimated on a basis of cumulative frequencies and the values on the continuum from 0 to 10 of the boundaries between the response options of the primary scale. The mean μ based on the parameters of this best fit beta distribution is then considered to be an estimator of the mean happiness in a population.

The Happiness Scale Interval Study

The Happiness Scale Interval Study, is designed to be used for survey questions on happiness using verbal response options. It is used to determine what degrees of happiness are denoted by such terms when used in particular questions and different languages. The method uses a web-based Scale Interval Recorder. First a description is given of this Scale Interval Recorder. Next, the differences with conventional scale transformation methods are discussed. This is followed by an exemplar application of the conventional methods and the Scale Interval Method to existing survey data. The results of these applications are mutually compared from the perspective of the comparability problem.

Analysis of Differences in Trends Among the Satisfied Few and the Dissatisfied Few

Scholars often want to know what the interrelations of the means in subgroups of the population are, for example from the perspective of inequality or marginalization. When using a discrete response scale with a limited number of response options it will be a difficult task to derive this information from the survey results. This difficulty can be overcome if the Reference Distribution Method is applied to these survey results. Once a continuous distribution has been found to estimate a population mean for life satisfaction over time, it is also possible to estimate the means for subgroups of a population such as the least satisfied and the most satisfied part. We use the topics satisfaction with life and satisfaction with how democracy works in Western Europe, Southern Europe and Eastern Europe to demonstrate how the Reference Distribution Method can be used for this purpose.