Maureen Schmitter-Edgecombe

The Ecological Validity of Neuropsychological Tests: A Review of the Literature on Everyday Cognitive Skills

Evaluating the ecological validity of neuropsychological tests has become an increasingly important topic over the past decade. In this paper, we provide a comprehensive review of the research on the ecological validity of neuropsychological tests, as it pertains to everyday cognitive skills. This review is presented in the context of several theoretical issues facing ecological validity research. Overall, the research suggests that many neuropsychological tests have a moderate level of ecological validity when predicting everyday cognitive functioning. The strongest relationships were noted when the outcome measure corresponded to the cognitive domain assessed by the neuropsychological tests. Several other factors that may moderate the degree of ecological validity established for neuropsychological tests are in need of further exploration. These factors include the effects of the population being tested, the approach utilized (verisimilitude vs. veridicality), the person completing the outcome measure (significant other vs. clinician), illness severity, and time from injury until evaluation. In addition, a standard measurement of outcome for each cognitive domain is greatly needed to allow for comparison across studies.

Discovering Activities to Recognize and Track in a Smart Environment

The machine learning and pervasive sensing technologies found in smart homes offer unprecedented opportunities for providing health monitoring and assistance to individuals experiencing difficulties living independently at home. In order to monitor the functional health of smart home residents, we need to design technologies that recognize and track activities that people normally perform as part of their daily routines. Although approaches do exist for recognizing activities, the approaches are applied to activities that have been preselected and for which labeled training data are available. In contrast, we introduce an automated approach to activity tracking that identifies frequent activities that naturally occur in an individuals routine. With this capability, we can then track the occurrence of regular activities to monitor functional health and to detect changes in an individuals patterns and lifestyle. In this paper, we describe our activity mining and tracking approach, and validate our algorithms on data collected in physical smart environments.

Assessing the Quality of Activities in a Smart Environment

OBJECTIVES Pervasive computing technology can provide valuable health monitoring and assistance technology to help individuals live independent lives in their own homes. As a critical part of this technology, our objective is to design software algorithms that recognize and assess the consistency of activities of daily living that individuals perform in their own homes. METHODS We have designed algorithms that automatically learn Markov models for each class of activity. These models are used to recognize activities that are performed in a smart home and to identify errors and inconsistencies in the performed activity. RESULTS We validate our approach using data collected from 60 volunteers who performed a series of activities in our smart apartment testbed. The results indicate that the algorithms correctly label the activities and successfully assess the completeness and consistency of the performed task. CONCLUSIONS Our results indicate that activity recognition and assessment can be automated using machine learning algorithms and smart home technology. These algorithms will be useful for automating remote health monitoring and interventions.

Characterizing multiple memory deficits and their relation to everyday functioning in individuals with mild cognitive impairment.

The authors evaluated multiple memory processes and explored their contributions to everyday functional limitations in persons with mild cognitive impairment (MCI). Participants included individuals with amnestic MCI, nonamnestic MCI, and healthy older adults. As expected, the amnestic MCI group performed more poorly than the control and nonamnestic MCI groups on a content memory measure. Both MCI groups, however, performed more poorly than controls on the noncontent memory measures of prospective memory, temporal order memory, and source memory. Informants also reported that the MCI groups were experiencing greater difficulty than controls completing instrumental activities of daily living (IADLs). Noncontent memory measures were found to make an independent contribution to IADL performances over and above that of content memory. These findings demonstrate that impairments in memory beyond the traditionally assessed content memory are present in individuals with amnestic MCI and with nonamnestic MCI. The results further show that these noncontent memory processes, which have been linked with executive functioning, play a role in supporting IADLs.

Recognizing independent and joint activities among multiple residents in smart environments.

The pervasive sensing technologies found in smart homes offer unprecedented opportunities for providing health monitoring and assistance to individuals experiencing difficulties living independently at home. A primary challenge that needs to be tackled to meet this need is the ability to recognize and track functional activities that people perform in their own homes and everyday settings. In this paper, we look at approaches to perform real-time recognition of Activities of Daily Living. We enhance other related research efforts to develop approaches that are effective when activities are interrupted and interleaved. To evaluate the accuracy of our recognition algorithms we assess them using real data collected from participants performing activities in our on-campus smart apartment testbed.

Memory remediation after severe closed head injury: notebook training versus supportive therapy.

This study evaluated the effectiveness of a 9-week memory notebook treatment for closed-head-injured (CHI) participants with documented memory deficits. Eight participants who had sustained a severe CHI more than 2 years earlier were allocated to receive either notebook training or supportive therapy. Memory outcome indicators, which differed in sensitivity to detect everyday memory failures (EMFs), were administered before treatment, immediately after treatment, and at a 6-month follow-up. At posttreatment, the notebook training group reported significantly fewer observed EMFs on a daily checklist measure than the supportive therapy group. Although in the same direction, this finding no longer reached significance at follow-up. No significant treatment effects were found for the laboratory-based memory measures at posttreatment or follow-up. Although the present results are to be considered preliminary because of the small sample size, they suggest that notebook training has the potential to help individuals compensate for everyday memory problems and that the methods used to measure training efficacy are important.

Cognitive Correlates of Functional Performance in Older Adults: Comparison of Self-Report, Direct Observation, and Performance-Based Measures

Neuropsychologists are often asked to answer questions about the effects of cognitive deficits on everyday functioning. This study examined the relationship between and the cognitive correlates of self-report, performance-based, and direct observation measures commonly used as proxy measures for everyday functioning. Participants were 88 community-dwelling, cognitively healthy older adults (age 50-86 years). Participants completed standardized neuropsychological tests and questionnaires, and performed eight activities of daily living (e.g., water plants, fill a medication dispenser) while under direct observation in a campus apartment. All proxy measures of everyday function were sensitive to the effects of healthy cognitive aging. After controlling for age, cognitive predictors explained a unique amount of the variance for only the performance-based behavioral simulation measure (i.e., Revised Observed Tasks of Daily Living). The self-report instrumental activities of daily living (IADL) and the performance-based everyday problem-solving test (i.e., EPT) did not correlate with each other; however, both were unique predictors of the direct observation measure. These findings suggest that neuropsychologists must be cautious in making predictions about the quality of everyday activity completion in cognitively healthy older adults from specific cognitive functions. The findings further suggest that a self-report of IADLs and the performance-based EPT may be useful measures for assessing everyday functional status in cognitively healthy older adults.

Automated Cognitive Health Assessment Using Smart Home Monitoring of Complex Tasks

One of the many services that intelligent systems can provide is the automated assessment of resident well-being. We hypothesize that the functional health of individuals, or ability of individuals to perform activities independently without assistance, can be estimated by tracking their activities using smart home technologies. In this paper, we introduce a machine-learning-based method to assess activity quality in smart homes. To validate our approach, we quantify activity quality for 179 volunteer participants who performed a complex, interweaved set of activities in our smart home apartment. We compare our automated assessment of task quality with direct observation scores. We also assess the ability of machine-learning techniques to predict the cognitive health of the participants based on these automated scores. We believe that this capability is an important step in understanding everyday functional health of individuals in their home environments.

Applications of Technology in Neuropsychological Assessment

Most neuropsychological assessments include at least one measure that is administered, scored, or interpreted by computers or other technologies. Despite supportive findings for these technology-based assessments, there is resistance in the field of neuropsychology to adopt additional measures that incorporate technology components. This literature review addresses the research findings of technology-based neuropsychological assessments, including computer- and virtual reality-based measures of cognitive and functional abilities. We evaluate the strengths and limitations of each approach, and examine the utility of technology-based assessments to obtain supplemental cognitive and behavioral information that may be otherwise undetected by traditional paper-and-pencil measures. We argue that the potential of technology use in neuropsychological assessment has not yet been realized, and continued adoption of new technologies could result in more comprehensive assessment of cognitive dysfunction and in turn, better informed diagnosis and treatments. Recommendations for future research are also provided.

Working memory and aging: A cross-sectional and longitudinal analysis using a self-ordered pointing task

Age-related declines in working memory performance have been associated with deficits in inhibition, strategy use, processing speed, and monitoring. In the current study, cross-sectional and longitudinal methodologies were used to investigate the relative contribution of these components to age-related changes in working memory. In Experiment 1, a sample of 140 younger and 140 older adults completed an abstract design version of the Self-Ordered Pointing Task modeled after Shimamura and Jurica (1994). Experiment 1 revealed that only processing speed and monitoring explained age differences in SOPT performance. Participants in Experiment 2 were 53 older adults who returned 4 years after the initial testing and 53 young adults. A task that assessed the ability to generate and monitor an internal series of responses as compared to an externally imposed series of responses was also administered. Experiment 2 replicated the key findings from Experiment 1 and provided some further evidence for age-related internal monitoring difficulties. Furthermore, the exploratory longitudinal analysis revealed that older age and lower intellectual abilities tended to be associated with poorer performance on the SOPT at Time 2.