R. John Irwin

Relativity of Judgements about Sound Amplitude and the Asymmetry of the Same-Different ROC:

The problem of deciding whether two things are the same or different in magnitude can be solved by judging one magnitude relative to the other, or by making absolute judgements about the magnitude of each. The shape of the resulting receiver operating characteristic depends on which solution is adopted. In order to obtain empirical receiver operating characteristics, we therefore had subjects rate their confidence that two tone amplitudes were the same or different. Four subjects each made 500 ratings of three differences in amplitude. The asymmetry in the obtained characteristics indicated that subjects made relative rather than absolute judgements of the amplitudes, despite the fact that making absolute judgements would lead to better performance on the task.

Perception of simple and complex visual stimuli: decision strategies and hemispheric differences in same-different judgments.

The accuracy with which observers could judge whether two visual stimuli were the same or different was measured with the rating method of detection theory. For judgments of whether two pictures referred to natural or manufactured things, the shape of the obtained receiver operating characteristic (ROC) was consistent with the observers adopting an optimal decision strategy. A similar result was found for judgments of complex but meaningless visual patterns. For judgments of whether two colours that differed along a simple sensory dimension were the same or different, however, the resulting ROC was consistent with the observers adopting a suboptimal differencing strategy. The accuracy of the judgments did not depend on the visual field to which the stimuli were presented.

Likelihood-ratio decision strategy for independent observations in thesame-different task: An approximation to the detection-theoretic model

An optimal decision strategy for deciding whether two things are the same or different is to adopt a likelihood-ratio criterion. The parametric equations for the receiver operating characteristic (ROC) based on the likelihood-ratio strategy when observations are independent are complicated; they require the numerical evaluation of a double integral. An approximation to the parametric equations for the likelihood-ratio strategy was developed. This approximation takes the form of a pair of equations that describe ROCs virtually indistinguishable from those of the full model.

Decision strategies and visual-field asymmetries in same-different judgments of word meaning

The accuracy with which observers judged whether two words belonged to the same semantic category was determined from a detection-theoretic analysis ofsame-different judgments. In Experiment 1, one word was presented centrally and the other word in either the left visual field (LVF) or the right visual field (RVF); in Experiment 2, both words were presented to either the LVF or the RVF. In order to obtain receiver-operating characteristics (ROCs) of performance, observers were asked to rate their confidence that the two words belonged to the same semantic category. Two models of the decision strategy were fitted to the obtained characteristics: a differencing model, in which the decision variable was the difference between the two observations; and an optimal model, in which each observation was judged in relation to a criterion. In both experiments, the optimal model provided a better fit than the differencing model to the obtained characteristics. Maximum-likelihood estimates of both the criterion-free parameter,d′, and the area under the operating characteristic,p(A), were greater for words presented in the RVF than for those presented in the LVF.

An area theorem for thesame-different experiment

If observers in a same-different experiment base their decisions on the absolute difference between observations on a trial, the area under the receiver operating characteristic equals the maximum proportion of correct decisions that an unbiased independent-observations observer could attain. Even though the differencing strategy is suboptimal,the area measure yields an index of optimal performance.

Identification of Mathematically Gifted Children in New Zealand

A popular way of identifying mathematically gifted children in New Zealand is to select those ranked at or above the 90th percentile on a mathematics multiple-choice test, the Progressive Achievement Test (PAT). Giftedness in mathematics can properly be defined as the ability to solve difficult mathematical problems. We therefore compared the performance of sixty-six children on a published problem-solving test and the PAT. Receiver Operating Characteristic analysis showed that, independent of any chosen percentile, the PATs accuracy at identifying mathematically gifted children was 78 per cent. Such a degree of accuracy will lead to many mathematically gifted children being overlooked, or many being mistakenly identified as gifted. Use of the PAT for this purpose is therefore not recommended.

Indices of response bias in the same-different experiment

Thesame-different experiment is popular for assessing perceptual performance, including the performance of people with neuropsychological deficits. Although the measurement of accuracy with this experiment is now well understood, the measurement of response bias remains problematic. Reformulating the decision space for the experiment can yield new bias indices that are analogous to, but numerically different from, those of the more familiar yesno experiment. Isobias curves that show how hit rate covaries with false alarm rate for constant bias but varying accuracy are presented for eight indices, and best-fitting parameters of the isobias functions are determined for a set of experimental data. The theoretical status of the bias indices and their relation to other formulations are reviewed.

Equivalence of the statistics for replicability and area under the ROC curve.

Two statistics, one recent and one well known, are shown to be equivalent. The recent statistic, p(rep), gives the probability that the sign of an experimental effect is replicable by an experiment of equal power. That statistic is equivalent to the well-known measure for the area under a receiver operating characteristic (ROC) curve for statistical power against significance level. Both statistics can be seen as exemplifying the area theorem of psychophysics.

Stability of memory for colour in context.

Memory for colours presented in isolation was compared with that for colours presented as part of a clip-art image or as part of a non-meaningful, Mondrian-like image. The area under the Receiver Operating Characteristic for deciding that two colours, presented at different delay intervals, were the same or different provided an index of memory for colour. The provision of a context reduced the decay of memory, regardless of whether the context was meaningful (clip-art images) or non-meaningful (Mondrian-like images). The result was seen as a generalisation of the auditory phenomenon of profile analysis, in which memory for the amplitude of a single component of a complex sound is more stable than that for the component in isolation.

Discriminability of electrocutaneous stimuli after topical anesthesia: Detection-theory measurement of sensitivity to painful stimuli

In three experiments on the psychophysical measurement of pain, electrocutaneous currents were applied to the volar surface of the forearm. In the first experiment, a conventional category scaling method was compared with the rating method of signal detection. The results of both methods were analyzed in detection-theory terms to derive receiver operating characteric curves and measures of the discriminability of adjacent currents. The rating method yielded larger discriminability values than the category scale did, and that method was therefore used in the subsequent experiments to examine the effect of a topical anesthetic on discriminability. When the stimuli were applied through surface electrodes, no effect of the topical anesthetic on discriminability was found, but when the stimuli were applied to a more localized area by intradermal needle electrodes, a dose-dependent effect of the anesthetic on discriminability occurred. For this experiment, the slope of the cumulative sensitivity function increased with increasing elapsed time since the removal of the anesthetic. This result is congruent with the theory that discriminability can serve as a measure of sensitivity to painful stimuli.