Efstathios Stavrakis

A psychophysical study of fixation behavior in a computer game

Prediction of gaze behavior in gaming environments can be a tremendously useful asset to game designers, enabling them to improve gameplay, selectively increase visual fidelity, and optimize the distribution of computing resources. The use of saliency maps is currently being advocated as the method of choice for predicting visual attention, crucially under the assumption that no specific task is present. This is achieved by analyzing images for low-level features such as motion, contrast, luminance, etc. However, the majority of computer games are designed to be easily understood and pose a task readily apparent to most players. Our psychophysical experiment shows that in a task-oriented context such as gaming, the predictive power of saliency maps at design time can be weak. Thus, we argue that a more involved protocol utilizing eye tracking, as part of the computer game design cycle, can be sufficiently robust to succeed in predicting fixation behavior of players.

Image-based stereoscopic painterly rendering

We present a new image-based stereoscopic painterly algorithm that we use to automatically generate stereoscopic paintings. Our work is motivated by contemporary painters who have explored the aesthetic implications of painting stereo pairs of canvases. We base our method on two real images, acquired from spatially displaced cameras. We derive a depth map by utilizing computer vision depth-from-stereo techniques and use this information to plan and render stereo paintings. These paintings can be viewed stereoscopically, in which case the pictorial medium is perceptually extended by the viewer to better suggest the sense of distance.

An empirical pipeline to derive gaze prediction heuristics for 3D action games

Gaze analysis and prediction in interactive virtual environments, such as games, is a challenging topic since the 3D perspective and variations of the viewpoint as well as the current task introduce many variables that affect the distribution of gaze. In this article, we present a novel pipeline to study eye-tracking data acquired from interactive 3D applications. The result of the pipeline is an importance map which scores the amount of gaze spent on each object. This importance map is then used as a heuristic to predict a users visual attention according to the object properties present at runtime. The novelty of this approach is that the analysis is performed in object space and the importance map is defined in the feature space of high-level properties. High-level properties are used to encode task relevance and other attributes, such as eccentricity, which may have an impact on gaze behavior. The pipeline has been tested with an exemplary study on a first-person shooter game. In particular, a protocol is presented describing the data acquisition procedure, the learning of different importance maps from the data, and finally an evaluation of the performance of the derived gaze predictors. A metric measuring the degree of correlation between attention predicted by the importance map and the actual gaze yielded clearly positive results. The correlation becomes particularly strong when the player is attentive to an in-game task.

Topological Sound Propagation with Reverberation Graphs

Reverberation graphs is a novel approach to estimate global soundpressure decay and auralize corresponding reverberation effects in interactive virtual environments. We use a 3D model to represent the geometry of the environment explicitly, and we subdivide it into a series of coupled spaces connected by portals. Off-line geometrical-acoustics techniques are used to precompute transport operators, which encode pressure decay characteristics within each space and between coupling interfaces. At run-time, during an interactive simulation, we traverse the adjacency graph corresponding to the spatial subdivision of the environment. We combine transport operators along different sound propagation routes to estimate the pressure decay envelopes from sources to the listener. Our approach compares well with off-line geometrical techniques, but computes reverberation decay envelopes at interactive rates, ranging from 12 to 100 Hz. We propose a scalable artificial reverberator that uses these decay envelopes to auralize reverberation effects, including room coupling. Our complete system can render as many as 30 simultaneous sources in large dynamic virtual environments.

Stereoscopic painting with varying levels of detail

We present an algorithm for generating automatically stereoscopic paintings with varying levels of detail. We describe our interactive system built around the algorithm to enable users to adjust the level of detail of the painting. In this context of interactivity we have modified our stereo painting algorithm, presented in previous work, in order to explore the idea of user-driven artistic level-of-detail selection and display. In particular, a stereo painting is composed by two canvases, one for each eye. These canvases contain multiple refining coarse-to-fine layers of brush strokes that compose the final painting. In past research, the underlying coarser layers are obscured and function only as the basis to progressively build the finer painting layers. In contrast, our interactive stereo viewing system enables the user to selectively toggle the visibility of finer strokes to reveal coarser representations of the artwork.

Image-based stereoscopic stylization

We present a method to generate stylized stereo imagery that effectively communicates shape and distance of the depicted scene objects. We use computer vision techniques to analyze real stereo image pairs. In particular, a region based stereo matching algorithm with symmetrical treatment of occlusions is used to extract a disparity map and successively the depth information of the scene. The reference image is color segmented for the purpose of color stylization and an algorithm combining intensity image edges and depth discontinuities is applied to depict dominant object contours in the image. We use disparity information to propagate stylized color segments to the second view together with the object outlining contours. The stylized image pairs are consistent across the two views and can be easily fused for stereoscopic viewing. The stereoscopic image fusion provides an extra dimension of depth that is absent on the individual images.

Gaze-to-Object Mapping during Visual Search in 3D Virtual Environments

Stimuli obtained from highly dynamic 3D virtual environments and synchronous eye-tracking data are commonly used by algorithms that strive to correlate gaze to scene objects, a process referred to as gaze-to-object mapping (GTOM). We propose to address this problem with a probabilistic approach using Bayesian inference. The desired result of the inference is a predicted probability density function (PDF) specifying for each object in the scene a probability to be attended by the user. To evaluate the quality of a predicted attention PDF, we present a methodology to assess the information value (i.e., likelihood) in the predictions of different approaches that can be used to infer object attention. To this end, we propose an experiment based on a visual search task, which allows us to determine the object of attention at a certain point in time under controlled conditions. We perform this experiment with a wide range of static and dynamic visual scenes to obtain a ground-truth evaluation dataset, allowing us to assess GTOM techniques in a set of 30 particularly challenging cases.

Computer generated stereoscopic artwork

The focus of this work is to investigate and simulate artistic techniques in stereoscopy that go beyond stereo photography, such as stereoscopic painting. We briefly discuss the topic from a stereo artists perspective and map some of our observations from traditional stereo techniques to the scientific domain, where we use them to tackle technical tasks involved in the generation of stereo artwork. We describe a framework that we use in our stereoscopic image-based non-photorealistic rendering algorithms, but it can be adopted by other single-view artistic image synthesis techniques in order to generate stereoscopic output.

Interactive tools for image-based stereoscopic artwork

In this paper we describe a set of interactive tools that we have built as an extension to our image-based stereoscopic non-photorealistic rendering system. The base system is capable of automatically turning stereoscopic input images to stereoscopic pictures that resemble artwork, including concept drawings, cartoons and paintings. The tools described here aim to complement the traditional stereoscopic viewing experience of the end-user by enabling him to interact with the perceived stereoscopic space. The observers of the generated artwork can easily enhance the perceived depth by manipulating the two artistic-looking projections while stereo viewing. The users can examine the stereoscopic artwork via the use of stereoscopic cursors, as well as explore the structure of multi-layered artwork by peeling away layers at different depths to reveal other layers, initially occluded.

Parameterized sketches from stereo images

In this paper we present an algorithm to automatically generate sketches from stereo image pairs. Stereo analysis is initially performed on the input stereo pair to estimate a dense depth map. An Edge Combination image is computed by localising object contour edges, as indicated by the depth map, within the intensity reference image. We then approximate these pixel-represented contours by devising a new parametric curve fitting algorithm. The algorithm successfully recovers a minimum number of control points required to fit a Bezier curve onto the pixel-edge dataset in the Edge Combination image. Experiments demonstrate how the Edge Combination algorithm, used for dominant edge extraction, can be combined with a curve fitting algorithm to automatically provide parameterized artistic sketches or concept drawings of a real scene.