Bryan L. Koenig

An emotion understanding framework for intelligent agents based on episodic and semantic memories

Emotional intelligence is the ability to process information about one’s own emotions and the emotions of others. It involves perceiving emotions, understanding emotions, managing emotions and using emotions in thought processes and in other activities. Emotion understanding is the cognitive activity of using emotions to infer why an agent is in an emotional state and which actions are associated with the emotional state. For humans, knowledge about emotions includes, in part, emotional experiences (episodic memory) and abstract knowledge about emotions (semantic memory). In accordance with the need for more sophisticated agents, the current research aims to increase the emotional intelligence of software agents by introducing and evaluating an emotion understanding framework for intelligent agents. The framework organizes the knowledge about emotions using episodic memory and semantic memory. Its episodic memory learns by storing specific details of emotional events experienced firsthand or observed. Its semantic memory is a lookup table of emotion-related facts combined with semantic graphs that learn through abstraction of additional relationships among emotions and actions from episodic memory. The framework is simulated in a multi-agent system in which agents attempt to elicit target emotions in other agents. They learn what events elicit emotions in other agents through interaction and observation. To evaluate the importance of different memory components, we run simulations with components “lesioned”. We show that our framework outperformed Q-learning, a standard method for machine learning.

An Automated Estimator of Image Visual Realism Based on Human Cognition

Assessing the visual realism of images is increasingly becoming an essential aspect of fields ranging from computer graphics (CG) rendering to photo manipulation. In this paper we systematically evaluate factors underlying human perception of visual realism and use that information to create an automated assessment of visual realism. We make the following unique contributions. First, we established a benchmark dataset of images with empirically determined visual realism scores. Second, we identified attributes potentially related to image realism, and used correlational techniques to determine that realism was most related to image naturalness, familiarity, aesthetics, and semantics. Third, we created an attributes-motivated, automated computational model that estimated image visual realism quantitatively. Using human assessment as a benchmark, the model was below human performance, but outperformed other state-of-the-art algorithms.

Real or Fake ?: human judgments about photographs and computer-generated images of faces

While efforts to create realistic images have generated break-throughs in computer graphics modeling, there has been little research to date on factors causing people to perceive images as real versus computer-generated (CG). The motivation of the current experiment is to begin investigating such factors. We showed both computer-graphics experts and laypersons real photos and CG images. Photo and CG images were of three types: original, modified to show only intrinsic reflectance components, and modified to show only intrinsic shading components (grayscale). Participants judged whether each image was a real photo or a CG image. Results showed that visual realism depends not only on image properties, but also on cognitive characteristics of viewers. Shading was especially crucial for visual realism. Color was also important. Experts outperformed laypersons, but their advantage was limited to grayscale images. This research at the interface between human cognition and computer vision is a starting point for investigating the factors underlying visual realism.

Using modified incremental chart parsing to ascribe intentions to animated geometric figures

People spontaneously ascribe intentions on the basis of observed behavior, and research shows that they do this even with simple geometric figures moving in a plane. The latter fact suggests that 2-D animations isolate critical information—object movement—that people use to infer the possible intentions (if any) underlying observed behavior. This article describes an approach to using motion information to model the ascription of intentions to simple figures. Incremental chart parsing is a technique developed in natural-language processing that builds up an understanding as text comes in one word at a time. We modified this technique to develop a system that uses spatiotemporal constraints about simple figures and their observed movements in order to propose candidate intentions or nonagentive causes. Candidates are identified via partial parses using a library of rules, and confidence scores are assigned so that candidates can be ranked. As observations come in, the system revises its candidates and updates the confidence scores. We describe a pilot study demonstrating that people generally perceive a simple animation in a manner consistent with the model.

Re-examining dominance of categories in impression formation: A test of dual-process models.

We carried out tests of the first 2 premises of the Continuum Model (CM) of impression formation (Fiske & Neuberg, 1990). These premises predict that category information will in general be more influential than noncategory information, and that the fit of noncategorical attributes with the category is a major determinant of the relative influence of these types of information. Using stimuli that included sets of (a) text items only, and (b) combinations of photos and text items, we found no support for these claims, even using alternative tests. In addition, many positive effects found in our analyses run counter to the predictions of the CM. We conclude that either significant portions of dual-process models (also, Brewer, 1988) are not applicable to many previously claimed scenarios of impression formation, or that although pieces of them may be roughly accurate, reasonable questions arise as to their predictive and discriminant validity.

Cross-race misaggregation: Its detection, a mathematical decomposition, and Simpson's paradox

Researchers sometimes aggregate data, such as combining resident data into state-level means. Doing so can sometimes cause valid individual-level data to be invalid at the group level. We focus on cross-race misaggregation, which can occur when individual-level data are confounded with race. We discuss such misaggregation in the context of Simpson’s Paradox and identify 4 diagnostic indicators: aggregated rates that correlate strongly with the relative size of one or more subgroup(s), unequal sample sizes across subgroups, unequal rates or mean values across subgroups, and aggregated rates that do not correlate with subgroup rates. To illustrate these diagnostic indicators, we decomposed data on the prevalence of sexually transmitted diseases (STDs) to confirm cross-race misaggregation in Parasite Stress U.S.A., an ostensible index of parasite prevalence known to be confounded with the proportion of African American residents per state.