Laura Mickes

Remember/know judgments probe degrees of recollection

Remembering and knowing are states of awareness that accompany the retrieval of facts, faces, and experiences from our past. Although originally intended to separate episodic from semantic memory, the dominant view today is that recollection-based decisions underlie remember responses, whereas familiarity-based decisions underlie know responses. Many functional magnetic resonance imaging (fMRI) studies as well as lesion studies have relied on the remember/know procedure to identify the neural correlates of recollection and familiarity. An implicit assumption of this approach is that know responses, which are thought to tap familiarity-based decisions, are devoid of recollection. We investigated this issue by using a source memory procedure and found that the accuracy of source recollection was significantly above chance for studied words that were declared to be old and known. Critically, this held true even when the source decision was made before the old/new decision (i.e., even after successful recollection had just occurred). Our results show that although recollection and familiarity may be different processes, the remember/know paradigm does not probe them directly. As such, dissociations involving remember/know judgments in fMRI studies and in studies involving amnesic patients should not be construed as dissociations between recollection and familiarity.

A direct test of the unequal-variance signal detection model of recognition memory

Analyses of the receiver operating characteristic (ROC) almost invariably suggest that, on a recognition memory test, the standard deviation of memory strengths associated with the lures (σlure) is smaller than that of the targets (σtarget). Often, σlure/σtarget ≈ 0.80. However, that conclusion is based on a model that assumes that the memory strength distributions are Gaussian in form. In two experiments, we investigated this issue in a more direct way by asking subjects to simply rate the memory strengths of targets and lures using a 20-point or a 99-point strength scale. The results showed that the standard deviation of the ratings made to the targets (starget) was, indeed, larger than the standard deviation of the ratings made to the lures (slure). Moreover, across subjects, the ratioslure/starget correlated highly with the estimate of σlure/σtarget obtained from ROC analysis, and both estimates were, on average, approximately equal to 0.80.

Recollection Is a Continuous Process Implications for Dual-Process Theories of Recognition Memory

Dual-process theory, which holds that recognition decisions can be based on recollection or familiarity, has long seemed incompatible with signal detection theory, which holds that recognition decisions are based on a singular, continuous memory-strength variable. Formal dual-process models typically regard familiarity as a continuous process (i.e., familiarity comes in degrees), but they construe recollection as a categorical process (i.e., recollection either occurs or does not occur). A continuous process is characterized by a graded relationship between confidence and accuracy, whereas a categorical process is characterized by a binary relationship such that high confidence is associated with high accuracy but all lower degrees of confidence are associated with chance accuracy. Using a source-memory procedure, we found that the relationship between confidence and source-recollection accuracy was graded. Because recollection, like familiarity, is a continuous process, dual-process theory is more compatible with signal detection theory than previously thought.

Receiver Operating Characteristic Analysis of Eyewitness Memory: Comparing the Diagnostic Accuracy of Simultaneous Versus Sequential Lineups

A police lineup presents a real-world signal-detection problem because there are two possible states of the world (the suspect is either innocent or guilty), some degree of information about the true state of the world is available (the eyewitness has some degree of memory for the perpetrator), and a decision is made (identifying the suspect or not). A similar state of affairs applies to diagnostic tests in medicine because, in a patient, the disease is either present or absent, a diagnostic test yields some degree of information about the true state of affairs, and a decision is made about the presence or absence of the disease. In medicine, receiver operating characteristic (ROC) analysis is the standard method for assessing diagnostic accuracy. By contrast, in the eyewitness memory literature, this powerful technique has never been used. Instead, researchers have attempted to assess the diagnostic performance of different lineup procedures using methods that cannot identify the better procedure (e.g., by computing a diagnosticity ratio). Here, we describe the basics of ROC analysis, explaining why it is needed and showing how to use it to measure the performance of different lineup procedures. To illustrate the unique advantages of this technique, we also report 3 ROC experiments that were designed to investigate the diagnostic accuracy of simultaneous versus sequential lineups. According to our findings, the sequential procedure appears to be inferior to the simultaneous procedure in discriminating between the presence versus absence of a guilty suspect in a lineup.

Measuring recollection and familiarity in the medial temporal lobe

Many recent studies have investigated how the structures of the medial temporal lobe (MTL) support recollection and familiarity, which are two processes widely thought to support recognition memory. The behavioral methods that are used to isolate recollection and familiarity in neuroimaging and lesion studies typically assume that recollection is a categorical process and not a continuous process. A categorical process is one that either occurs or does not occur for a particular test item (yielding high confidence and high accuracy when it does occur), whereas a continuous process is one that comes in degrees (yielding varying degrees of confidence and accuracy). Studies suggesting that the hippocampus selectively supports the recollection process (such as those that use the Remember/Know procedure or rely on Receiver Operating Characteristic analysis) generally depend on the categorical view of recollection, but much recent evidence suggests that recollection is a continuous process. If recollection is a continuous process (i.e., if recollection comes in degrees), then evidence that has been taken to mean that the hippocampus selectively supports recollection is also compatible with the idea that the hippocampus supports both recollection and familiarity. We suggest that an alternative method can be used to effectively investigate recollection and familiarity in the MTL, one that is valid whether recollection is a categorical or a continuous process.

Evaluating Eyewitness Identification Procedures Using Receiver Operating Characteristic Analysis

Eyewitness identification is a pivotal issue in applied research because, in practice, a correct identification can help to remove a dangerous criminal from society, but a false identification can lead to the erroneous conviction of an innocent suspect. Consequently, psychologists have tried to ascertain the best procedures for collecting identification evidence, evaluating them using measures based on the ratio of correct to false identification rates. Unfortunately, ratio-based measures are ambiguous because they change systematically as a function of a witness’s willingness to choose. In other words, a measure thought to index discriminability is instead fully confounded with response bias. A better method involves constructing receiver operating characteristic (ROC) curves. Using ROC curves, researchers can trace out discriminability across levels of response bias for each procedure. We illustrate the shortcomings of ratio-based measures and demonstrate why ROC analysis is required. In recent studies, researchers comparing simultaneous and sequential lineup procedures using ROC analyses have provided no evidence for the sequential superiority effect and instead have shown that the simultaneous procedure may be diagnostically superior. It is not yet clear which lineup procedure will prove to be generally superior, but it is clear that ROC analysis is the only way to make that determination.

Progressive impairment on neuropsychological tasks in a longitudinal study of preclinical Alzheimer's disease.

Previous research suggests that patients with Alzheimers disease exhibit cognitive impairment in the years preceding a clinical diagnosis. Memory impairments are particularly pronounced, but the relative degree to which other cognitive functions are impaired and the speed with which they decline during the preclinical years remains unclear. The authors report a detailed neuropsychological evaluation of 11 patients over the course of 3 years up to and including the 1st year of nonnormal diagnosis. The results suggest that performance falls off rapidly in all areas of cognitive functioning but that abilities thought to be subserved by the medial and lateral temporal lobes (episodic and semantic memory, respectively) appear to be substantially more impaired than those abilities thought to be subserved by the frontal lobes.

The Field of Eyewitness Memory Should Abandon Probative Value and Embrace Receiver Operating Characteristic Analysis.

Clark (2012) highlights an important issue that has received inadequate attention in the eyewitness memory literature: lineup procedures that reduce the false identification rate (a desirable effect) often tend to reduce the correct identification rate as well (an undesirable effect). Determining which procedure is diagnostically superior under those conditions is not easy. Clark (2012) showed that the procedure with the lower false identification rate could be associated with higher overall costs to society once costs and benefits are both taken into consideration. Beyond the issue of cost, we argue that Clark’s (2012) observation has far reaching implications for evaluating the diagnostic performance of a lineup procedure. Specifically, the field of eyewitness memory has attempted to differentiate between lineup procedures by using various measures of probative value (such as the diagnosticity ratio). However, contrary to intuition, probative value is not a relevant consideration. Instead, lineup procedures should be compared using receiver operating characteristic analysis, as is routinely done in other applied fields (such as radiology).

A signal-detection-based diagnostic-feature-detection model of eyewitness identification.

The theoretical understanding of eyewitness identifications made from a police lineup has long been guided by the distinction between absolute and relative decision strategies. In addition, the accuracy of identifications associated with different eyewitness memory procedures has long been evaluated using measures like the diagnosticity ratio (the correct identification rate divided by the false identification rate). Framed in terms of signal-detection theory, both the absolute/relative distinction and the diagnosticity ratio are mainly relevant to response bias while remaining silent about the key issue of diagnostic accuracy, or discriminability (i.e., the ability to tell the difference between innocent and guilty suspects in a lineup). Here, we propose a signal-detection-based model of eyewitness identification, one that encourages the use of (and helps to conceptualize) receiver operating characteristic (ROC) analysis to measure discriminability. Recent ROC analyses indicate that the simultaneous presentation of faces in a lineup yields higher discriminability than the presentation of faces in isolation, and we propose a diagnostic feature-detection hypothesis to account for that result. According to this hypothesis, the simultaneous presentation of faces allows the eyewitness to appreciate that certain facial features (viz., those that are shared by everyone in the lineup) are non-diagnostic of guilt. To the extent that those non-diagnostic features are discounted in favor of potentially more diagnostic features, the ability to discriminate innocent from guilty suspects will be enhanced.

Recollection Can Be Weak and Familiarity Can Be Strong

The remember-know procedure is widely used to investigate recollection and familiarity in recognition memory, but almost all of the results obtained with that procedure can be readily accommodated by a unidimensional model based on signal-detection theory. The unidimensional model holds that remember judgments reflect strong memories (associated with high confidence, high accuracy, and fast reaction times), whereas know judgments reflect weaker memories (associated with lower confidence, lower accuracy, and slower reaction times). Although this is invariably true on average, a new 2-dimensional account (the continuous dual-process model) suggests that remember judgments made with low confidence should be associated with lower old-new accuracy but higher source accuracy than know judgments made with high confidence. We tested this prediction--and found evidence to support it--using a modified remember-know procedure in which participants were first asked to indicate a degree of recollection-based or familiarity-based confidence for each word presented on a recognition test and were then asked to recollect the color (red or blue) and screen location (top or bottom) associated with the word at study. For familiarity-based decisions, old-new accuracy increased with old-new confidence, but source accuracy did not (suggesting that stronger old-new memory was supported by higher degrees of familiarity). For recollection-based decisions, both old-new accuracy and source accuracy increased with old-new confidence (suggesting that stronger old-new memory was supported by higher degrees of recollection). These findings suggest that recollection and familiarity are continuous processes and that participants can indicate which process mainly contributed to their recognition decisions.