Natalie A. Obrecht

Intuitivet tests: Lay use of statistical information

Normatively, a statistical pairwise comparison is a function of the mean, standard deviation (SD), and sample size of the data. In our experiment, 203 undergraduates compared product pairs and judged their confidence that one product was better than the other. We experimentally manipulated (within subjects) the average product ratings, the number of raters (sample size), and theSD of the ratings. Each factor had two levels selected, so that the same change in statistical power resulted from moving from the low to the high level. We also manipulated (between subjects) whether subjects were given only the product rating data as summarized in a statistical format or the summaries plus the raw ratings. Subjects gave the most weight to mean product ratings, less weight to sample size, and very little weight toSD. Providing subjects with raw data did not increase their use of sample size andSD, as predicted.

An encounter frequency account of how experience affects likelihood estimation

When making judgments, people often favor information received from a few individual sources over largesample statistical data. Individual information is usually acquired piece by piece, whereas statistical information combines many observations into a single summary. We examined whether this difference in the frequency of encounters affects how data are weighted. In two experiments, subjects read statistical information indicating an event to be rare and contrasting information from individual cases suggesting the event to be common. We controlled whether the individual cases were summarized into a single summary like statistical information, or presented serially, case by case. Subjects’ estimates of event frequencies were higher when the individual cases were presented in serial, rather than summarized, format. A third study demonstrated that subjects treat each data sample as an instance, and do not weight according to sample size. These results support the conclusion that people weight information according to encounter frequency.

Laypeople do use sample variance: The effect of embedding data in a variance-implying story

When using sample data to decide whether two populations differ, laypeople attend to the difference between group means, but largely overlook within-group variability (Obrecht, Chapman, & Gelman, 2007). We show, first, that laypeople know about and use story-implied variability when making pairwise comparisons. Then we demonstrate that participants’ sensitivity to variance in a dataset is boosted when presented in a context that implies consistent variance information. Statistical data were couched in stories about electrical conductivity measurements obtained from element samples (low-variability category) or body weight measurements from samples of peoples (high-variability category). We manipulated, between participants, whether the data variance matched or mismatched the story-implied variability. Participants who received data in a matching context showed high sensitivity to variance, while those in the mismatching condition did not. Laypeople use statistical data to make reasonable inferences when those data are provided in a context that makes sense.

Sample size weighting follows a curvilinear function.

Previous research is mixed regarding whether laypeople are sensitive to sample size. Here the author argues that this is in part because sample size sensitivity follows a curvilinear function with decreasing sensitivity as sample size become larger. This functional form reconciles apparent discrepancies in the literature, accounting for results where sample size is greatly attended to or nearly overlooked. The curvilinear form is found across confidence and estimation tasks that have square-root and linear normative standards. Thus, although people intuitively know that larger samples provide more reliable information, they do not modulate how they weight sample size in different circumstances. Further, individuals higher in numeracy show greater sensitivity to sample size than those lower numeracy (i.e., have a steeper curvilinear slope), but still underweight it relative to normative standards. Providing raw data can boost sample size use for individuals who are lower in numerical ability.