Elena A. Fedorovskaya

Aesthetics and Emotions in Images

In this tutorial, we define and discuss key aspects of the problem of computational inference of aesthetics and emotion from images. We begin with a background discussion on philosophy, photography, paintings, visual arts, and psychology. This is followed by introduction of a set of key computational problems that the research community has been striving to solve and the computational framework required for solving them. We also describe data sets available for performing assessment and outline several real-world applications where research in this domain can be employed. A significant number of papers that have attempted to solve problems in aesthetics and emotion inference are surveyed in this tutorial. We also discuss future directions that researchers can pursue and make a strong case for seriously attempting to solve problems in this research domain.

Chroma variations and perceived quality of color images of natural scenes

Transformations of natural images in the per- ulated with respect to luminance and chromaticity in order ceptually uniform CIELUV color space have been investi- to improve the display of color. Usually, some implicit gated with respect to perceptual image quality. To this assumptions are made to limit the almost infinite number end, digitized color images of four natural scenes were of possible manipulations. One of these assumptions is described on the basis of their color point distributions that only global changes need to be considered. This stems in the CIELUV color space. A new set of images was from the notion that images are coherent in the sense that created by varying the chroma value of each pixel while different parts of the same image share luminance and the lightness and hue angle were kept constant. The spectral properties due to the fact that they belong to the chroma was changed in two different ways: (1) through same object or are illuminated by the same light source. the addition or subtraction of the same amount of chroma This assumption underlies global reproduction techniques to or from the chroma value of each pixel; (2) through in which the same color operation is applied to all pixels, multiplication of the chroma value of each pixel by a irrespective of the content of the image. 1,2 Another as- constant. In three experiments, subjects judged the per- sumption is that the optimum image equals the original ceptual quality, colorfulness, and naturalness of the im- ( real-life ) scene. Hence, all efforts should be directed ages on a ten-point numerical category scale. The results towards finding ways of approaching this equality as indicate that colorfulness is the main perceptual attribute closely as possible. underlying image quality when chroma varies. Colorful- In this article, a different constraint on color reproduc- ness itself was found to depend on both the average tion is proposed, namely, the experienced naturalness of chroma and its variability. In general, the subjects pre- an image. This choice is motivated by the argument that ferred slightly more colorful images to the original ones. constraints should be formulated not in the physical, but The perceptual quality of the images was found to be in the psychological domain, the reason being that image closely related to the naturalness of the images.q 1997 quality judgments are based on a comparison between the

Capturing and sharing memories in a virtual world

Virtual worlds (VWs) such as Second Life® (SL) contain a rich social culture where people engage in a multitude of experiences much like real life. With this comes the need to capture and share memories with others. To understand what tools people use to accomplish this and what limitations they may face, we conducted interviews with participants in SL. Our results identify two clusters of users - Casuals and Lifers - who differed in the ways in which they captured and shared memories. Here we describe the use of photos, landmarks, friend lists, and conversation logs. We also show how a lack of real life physical and social constraints in the VW affects user routines, and, in some cases, how it does not. This suggests design directions for memory tools in the VW and also real life that break the bounds of current everyday practice.

The effect of familiarity on perceived interestingness of images

We present an exploration of familiarity as a meaningful dimension for the individualized adaptation of media-rich interfaces. In this paper, we investigate in particular the effect of digital images personalized for familiarity on users’ perceived interestingness. Two dimensions of familiarity, facial familiarity and familiarity with image context, are manipulated. Our investigation consisted of three studies: the first two address how morphing technology can be used to convey meaningful familiarity, and the third studies the effect of such familiarity on users’ sense of interestingness. Four levels of person familiarity varying in degree of person knowledge, and two levels of context familiarity varying in frequency of exposure, were considered: Self, Friend, Celebrity, and Stranger in Familiar and Unfamiliar contexts. Experimental results showed significant main effects of context and person familiarity. Our findings deepen understanding of the critical element of familiarity in HCI and its relationship to the interestingness of images, and can have great impact for the design of media-rich systems.

Subjective matters: from image quality to image psychology

From the advent of digital imaging through several decades of studies, the human vision research community systematically focused on perceived image quality and digital artifacts due to resolution, compression, gamma, dynamic range, capture and reproduction noise, blur, etc., to help overcome existing technological challenges and shortcomings. Technological advances made digital images and digital multimedia nearly flawless in quality, and ubiquitous and pervasive in usage, provide us with the exciting but at the same time demanding possibility to turn to the domain of human experience including higher psychological functions, such as cognition, emotion, awareness, social interaction, consciousness and Self. In this paper we will outline the evolution of human centered multidisciplinary studies related to imaging and propose steps and potential foci of future research.

Ambiguities in colour constancy and shape from shading

A new visual phenomenon—called the AMBEGUJAS phenomenon—is presented, together with some descriptive data from two initial exploratory experiments. The phenomenon is basically one of shape from shading, but ambiguous as to both shape and colour. There are two spontaneously alternating and mutually exclusive perceived 3-D shapes, and—as the most surprising observation—the colour impressions of these two shapes are markedly different. The stimulus situation is very simple with two differently coloured illuminations (with sharp edges) adjacently cast onto a flat, grey striped surface. In one 3-D shape almost the whole chromatic content disappears, and the surface goes towards its veridically grey colour. In the other the perceived object assumes the two illumination colours as clear surface colours. The decolorised percept is interpreted as a striking example of colour constancy: a perceptual solution with the classical ‘discounting of the illuminant’. Experiments show that the phenomenon is robust and appears in varying display layouts and different combinations of chromatic illuminations.

Colourfulness judgments of natural scenes

Abstract The purpose of this study was to examine the relationship between colourfulness judgments of images of natural scenes and statistical parameters of the chroma distribution over the images in the CIE 1976 L∗u∗v∗ (CIELUV) colour space. A comparative analysis of within-scenes and between-scenes colourfulness assessments was performed. Images were created by varying chroma in the CIELUV colour space while lightness and hue-angle were kept constant. Experiment 1 shows the results of a multidimensional analysis of the differences-scaling in perceived colourfulness. Experiment 2 includes a unidimensional investigation of the direct-scaling of the magnitude of perceived colourfulness. Both experiments confirm that colourfulness judgments highly correlate with a linear combination of the mean of the distribution of chroma values and its standard deviation. That was also verified in Experiment 3 with “Mondrian”-like stimuli (spatially scrambled versions of the images of natural scenes). Colourfulness judgments of natural scenes, but not “Mondrian”-like scenes, brought out systematic differences between observers. In order to interpret and quantify these differences, a model of individual strategies in colourfulness judgments is developed and discussed.

Identifying image preferences based on demographic attributes

The intent of this study is to determine what sorts of images are considered more interesting by which demographic groups. Specifically, we attempt to identify images whose interestingness ratings are influenced by the demographic attribute of the viewer’s gender. To that end, we use the data from an experiment where 18 participants (9 women and 9 men) rated several hundred images based on “visual interest” or preferences in viewing images. The images were selected to represent the consumer “photo-space” - typical categories of subject matter found in consumer photo collections. They were annotated using perceptual and semantic descriptors. In analyzing the image interestingness ratings, we apply a multivariate procedure known as forced classification, a feature of dual scaling, a discrete analogue of principal components analysis (similar to correspondence analysis). This particular analysis of ratings (i.e., ordered-choice or Likert) data enables the investigator to emphasize the effect of a specific item or collection of items. We focus on the influence of the demographic item of gender on the analysis, so that the solutions are essentially confined to subspaces spanned by the emphasized item. Using this technique, we can know definitively which images’ ratings have been influenced by the demographic item of choice. Subsequently, images can be evaluated and linked, on one hand, to their perceptual and semantic descriptors, and, on the other hand, to the preferences associated with viewers’ demographic attributes.