Ben Jann

Sensitive Questions in Online Surveys: Experimental Results for the Randomized Response Technique (RRT) and the Unmatched Count Technique (UCT)

Gaining valid answers to so-called sensitive questions is an age-old problem in survey research. Various techniques have been developed to guarantee anonymity and minimize the respondent’s feelings of jeopardy. Two such techniques are the randomized response technique (RRT) and the unmatched count technique (UCT). In this study the authors evaluate the effectiveness of different implementations of the RRT (using a forced-response design) in a computer-assisted setting and also compare the use of the RRT to that of the UCT. The techniques are evaluated according to various quality criteria, such as the prevalence estimates they provide, the ease of their use, and respondent trust in the techniques. The results indicate that the RRTs are problematic with respect to several domains, such as the limited trust they inspire and nonresponse, and that the RRT estimates are unreliable due to a strong false no bias, especially for the more sensitive questions. The UCT, however, performed well compared to the RRTs on all the evaluated measures. The authors conclude that the UCT is a promising alternative to RRT in self-administered surveys and that future research should be directed toward evaluating and improving the technique.

Estimating Heterogeneous Treatment Effects with Observational Data

Individuals differ not only in their background characteristics but also in how they respond to a particular treatment, intervention, or stimulation. In particular, treatment effects may vary systematically by the propensity for treatment. In this paper, we discuss a practical approach to studying heterogeneous treatment effects as a function of the treatment propensity, under the same assumption commonly underlying regression analysis: ignorability. We describe one parametric method and two nonparametric methods for estimating interactions between treatment and the propensity for treatment. For the first method, we begin by estimating propensity scores for the probability of treatment given a set of observed covariates for each unit and construct balanced propensity score strata; we then estimate propensity score stratum-specific average treatment effects and evaluate a trend across them. For the second method, we match control units to treated units based on the propensity score and transform the data into treatment-control comparisons at the most elementary level at which such comparisons can be constructed; we then estimate treatment effects as a function of the propensity score by fitting a nonparametric model as a smoothing device. For the third method, we first estimate nonparametric regressions of the outcome variable as a function of the propensity score separately for treated units and for control units and then take the difference between the two nonparametric regressions. We illustrate the application of these methods with an empirical example of the effects of college attendance on women’s fertility.

Reputation formation and the evolution of cooperation in anonymous online markets

Theoretical propositions stressing the importance of trust, reciprocity, and reputation for cooperation in social exchange relations are deeply rooted in classical sociological thought. Today’s online markets provide a unique opportunity to test these theories using unobtrusive data. Our study investigates the mechanisms promoting cooperation in an online-auction market where most transactions can be conceived as one-time-only exchanges. We first give a systematic account of the theoretical arguments explaining the process of cooperative transactions. Then, using a large dataset comprising 14,627 mobile phone auctions and 339,517 DVD auctions, we test key hypotheses about the effects of traders’ reputations on auction outcomes and traders’ motives for leaving feedback. Our statistical analyses show that sellers with better reputations have higher sales and obtain higher prices. Furthermore, we observe a high rate of participation in the feedback system, which is largely consistent with strong reciprocity—a predisposition to unconditionally reward (or punish) one’s interaction partner’s cooperation (or defection)—and altruism—a predisposition to increase one’s own utility by elevating an interaction partner’s utility. Our study demonstrates how strong reciprocity and altruism can mitigate the free-rider problem in the feedback system to create reputational incentives for mutually beneficial online trade.

Social determinants of sex differences in disability among older adults: a multi-country decomposition analysis using the World Health Survey

IntroductionWomen represent a growing proportion of older people and experience increasing disability in their longer lives. Using a universally agreed definition of disability based on the International Classification of Functioning, Disability and Health, this paper examines how, apart from age, social and economic factors contribute to disability differences between older men and women.MethodsWorld Health Survey data were analyzed from 57 countries drawn from all income groups defined by the World Bank. The final sample comprises 63638 respondents aged 50 and older (28568 males and 35070 females). Item Response Theory was applied to derive a measure of disability which ensured cross country comparability. Individuals with scores at or above a threshold score were those who experienced significant difficulty in their everyday lives, irrespective of the underlying etiology. The population was then divided into “disabled” vs. “not disabled”. We firstly computed disability prevalence for males and females by socio-demographic factors, secondly used multiple logistic regression to estimate the adjusted effects of each social determinant on disability for males and females, and thirdly used a variant of the Blinder-Oaxaca decomposition technique to partition the measured inequality in disability between males and females into the “explained” part that arises because of differences between males and females in terms of age and social and economic characteristics, and an “unexplained” part attributed to the differential effects of these characteristics.ResultsPrevalence of disability among women compared with men aged 50+ years was 40.1% vs. 23.8%. Lower levels of education and economic status are associated with disability in women and men. Approximately 45% of the sex inequality in disability can be attributed to differences in the distribution of socio-demographic factors. Approximately 55% of the inequality results from differences in the effects of the determinants.ConclusionsThere is an urgent need for data and methodologies that can identify how social, biological and other factors separately contribute to the health decrements facing men and women as they age. This study highlights the need for action to address social structures and institutional practices that impact unfairly on the health of older men and women.

Plagiarism in Student Papers: Prevalence Estimates Using Special Techniques for Sensitive Questions

Summary This article evaluates three different questioning techniques for measuring the prevalence of plagiarism in student papers: the randomized response technique (RRT), the item count technique (ICT), and the crosswise model (CM). In three independent experimental surveys with Swiss and German university students as subjects (two web surveys and a survey using paper and- pencil questionnaires in a classroom setting), each of the three techniques is compared to direct questioning and evaluated based on the “more-is-better” assumption. According to our results the RRT and the ICT failed to reduce social desirability bias in self-reports of plagiarism. In contrast, the CM was more successful in eliciting a significantly higher rate of reported sensitive behavior than direct questioning. One reason for the success of the CM, we believe, is that it overcomes the “self-protective no” bias known from the RRT (and which may also be a potential problem in the ICT).We find rates of up to 22 percent of students who declared that they ever intentionally adopted a passage from someone else’s work without citing it. Severe plagiarism such as handing in someone else’s paper as one’s own, however, seems to be less frequent with rates of about 1 to 2 percent.

Benford's Law and Fraud Detection. Facts and Legends

Abstract Is Benford’s law a good instrument to detect fraud in reports of statistical and scientific data? For a valid test, the probability of ‘false positives’ and ‘false negatives’ has to be low. However, it is very doubtful whether the Benford distribution is an appropriate tool to discriminate between manipulated and non-manipulated estimates. Further research should focus more on the validity of the test and test results should be interpreted more carefully.

Estimating the Prevalence of Illicit Drug Use Among Students Using the Crosswise Model

Objective: The aim of our study is to compare the prevalence of illicit drug use estimated through a technique referred to as the “crosswise model” (CM) with the results from conventional direct questioning (DQ). Method: About 1,500 students from Tehran University of Medical Sciences 2009–2010 were first interviewed by DQ and, then three months later, by the CM. Result: The CM yielded significantly higher estimates than DQ for lifetime prevalence of use of any illicit drug (CM = 20.2%,DQ = 3.0%, p < .001) and for lifetime prevalence of use of opium or its residue (CM = 13.6%, DQ = 1.0%, p < .001). Also, for use of any illicit drug in the last month and use of opium or its residue in the last month, the CM yielded higher point estimates than DQ, although these differences were not significant (any drug: CM = 1.5%, DQ = 0.2%, p = .66; opium: CM = 3.8%, DQ = 0.0%, p = .21). Conclusion: Our findings suggest that the CM is a fruitful data collection method for sensitive topics such as substance abuse.

A recruitment method to obtain community samples of children for survey research in Switzerland

In conducting epidemiological surveys of children in Switzerland, it is often very difficult to recruit large, representative community samples. To date, most large epidemiological surveys on children have used convenience samples, like school children (Jeannin et al. 2005); (Kuntsche and Kuendig 2006); (Steinhausen et al. 1998). One major limitation of school surveys is that certain information about the family and parents is often unavailable. Unlike other countries such as the USA, Switzerland has no so-called ‘census tracts’ (geographic regions defined for the purpose of taking a census) that can be used as sampling units. Most Swiss epidemiological surveys of adults have adopted two-stage sampling, selecting households by address or telephone number. However, information on household structure is not available before contacting the families. After calling a particular number or selecting a certain address, a designated person, usually of a particular age and/or gender, is chosen to participate. However, this method is impractical for studies on children, due to the narrow age range. When we only want to sample children, and especially children of a particular age (e.g. those 10–14 years old), we must screen more households to achieve a desired sample size. This Hints & Kinks presents a method for recruiting random community samples of children through local political municipalities (German: Gemeinden/French: communes/Italian: comuni) in Switzerland. Municipalities are the smallest governmental division in Switzerland. Their number has gradually decreased (from 3,095 in 1960 to 2,596 in 2010), mainly because smaller municipalities have found it difficult to absorb the necessary administrative costs of civil services. The population size of a municipality varies from less than 20 to more than 3,00,000 for a city like Zurich (Bundesamt für Statistik 2010). Every resident in Switzerland, regardless of nationality, must register in their municipality; consequently, municipalities have the most complete socio-demographic information and could be used as a basic sampling unit. We conducted a pilot study in June 2007 at Canton of Zurich to test this method of selecting random samples through municipalities. The underlying purpose of the study was to estimate the prevalence of Swiss children with special healthcare needs.

Diagnostik von Regressionsschätzungen bei kleinen Stichproben (mit einem Exkurs zu logistischer Regression)

Wie alle anderen statistischen Verfahren konzentriert sich auch die Methode der Regression nur auf die Analyse ausgewahlter Aspekte vorliegenden Datenmaterials. Entsprechend sind zu gegebenen Regressionsergebnissen ganz unterschiedliche Datenkonstellationen denkbar, wovon aber fur die Interpretation der Ergebnisse nicht alle unproblematisch sind. So besteht besonders bei kleinen Stichproben die Gefahr, dass die Regressionsschatzung entscheidend von einzelnen Extremwerten abhangt, was die Verlasslichkeit der daraus abgeleiteten Schlussfolgerungen beeintrachtigt. In diesem Beitrag werden deshalb anhand von Beispielen einige einfache grafische und formale Instrumente zur Diagnose einflussreicher Datenpunkte in der linearen und logistischen Regression vorgestellt, die im Prozess der Datenanalyse standardmasig angewendet werden sollten. Weiterhin werden nach Identifikation „atypischer“ Datenpunkte zu verfolgende Analysestrategien diskutiert.

Comment: Earnings Returns to Education in Urban China: A Note on Testing Differences among Groups

are higher in the market sector than in the state sector of China’s transition economy is commonly attributed to the “more efficiently operating market mechanisms” with respect to the valuation of human capital (Wu and Xie 2003:426). However, Wu and Xie “challenge the prevailing wisdom that education is necessarily more highly rewarded in the market sector” (p. 425). Basing their conclusion on an empirical analysis of this issue, Wu and Xie instead state that the market does not reward educational investment differentially. Unfortunately, Wu and Xie’s conclusion appears premature. As we illustrate later, their conclusion is problematic mainly because of a general difficulty in the interpretation of single pairwise tests in the analysis of the differences among three or more groups. The results of such comparisons can be ambiguous if the tests used lack sufficient statistical power. Although Wu and Xie compare the earnings returns to education among three different types of workers, they test only two of the three possible group differences. Because the results of the omitted test are in fact at odds with the other results, the viewpoint of Wu and Xie is biased and leads them to an unjustified conclusion. This discussion is organized as follows. First, we briefly summarize Wu and Xie’s argument and hypotheses and then replicate their analysis to illustrate its shortcomings. We next derive an alternative answer to their research question while providing some general advice on how to address the problem of multigroup comparisons.