Jakob Suchan

Grounding Dynamic Spatial Relations for Embodied (Robot) Interaction

This paper presents a computational model of the processing of dynamic spatial relations occurring in an embodied robotic interaction setup. A complete system is introduced that allows autonomous robots to produce and interpret dynamic spatial phrases (in English) given an environment of moving objects. The model unites two separate research strands: computational cognitive semantics and on commonsense spatial representation and reasoning. The model for the first time demonstrates an integration of these different strands.

Perceptual Narratives of Space and Motion for Semantic Interpretation of Visual Data

We propose a commonsense theory of space and motion for the high-level semantic interpretation of dynamic scenes. The theory provides primitives for commonsense representation and reasoning with qualitative spatial relations, depth profiles, and spatio-temporal change; these may be combined with probabilistic methods for modelling and hypothesising event and object relations. The proposed framework has been implemented as a general activity abstraction and reasoning engine, which we demonstrate by generating declaratively grounded visuo-spatial narratives of perceptual input from vision and depth sensors for a benchmark scenario.

The geometry of a scene: On deep semantics for visual perception driven cognitive film, studies

We present a general computational narrative model encompassing primitives of space, time, and motion from the viewpoint of deep knowledge representation and reasoning about visuo-spatial dynamics, and (eye-tracking based) visual perception of the moving image. The declarative model, implemented within constraint logic programming, integrates knowledge-based qualitative reasoning (e.g., about object / character placement, scene structure) with state of the art computer vision methods for detecting, tracking, and recognition of people, objects, and cinematographic devices such as cuts, shot types, types of camera movement. A key feature is that primitives of the theory - things, time, space and motion predicates, actions and events, perceptual objects (e.g., eye-tracking / gaze points, regions of attention etc) - are available as first-class objects with deep semantics suited for inference and query from the viewpoint of analytical Q&A or studies in visual perception. We present the formal framework and its implementation in the context of a large-scale experiment concerned with analysis of visual perception and reception of the moving image in the context of cognitive film studies.

Deep Semantic Abstractions of Everyday Human Activities

We propose a deep semantic characterization of space and motion categorically from the viewpoint of grounding embodied human-object interactions. Our key focus is on an ontological model that would be adept to formalisation from the viewpoint of commonsense knowledge representation, relational learning, and qualitative reasoning about space and motion in cognitive robotics settings. We demonstrate key aspects of the space & motion ontology and its formalization as a representational framework in the backdrop of select examples from a dataset of everyday activities. Furthermore, focussing on human-object interaction data obtained from RGBD sensors, we also illustrate how declarative (spatio-temporal) reasoning in the (constraint) logic programming family may be performed with the developed deep semantic abstractions.

Probabilistic Spatial Reasoning in Constraint Logic Programming

In this paper we present a novel framework and full implementation of probabilistic spatial reasoning within a Logic Programming context. The crux of our approach is extending Probabilistic Logic Programming (based on distribution semantics) to support reasoning over spatial variables via Constraint Logic Programming. Spatial reasoning is formulated as a numerical optimisation problem, and we implement our approach within ProbLog 1. We demonstrate a range of powerful features beyond what is currently provided by existing probabilistic and spatial reasoning tools.

Declarative Reasoning about Space and Motion with Video

We present a commonsense theory of space and motion for representing and reasoning about motion patterns in video data, to perform declarative (deep) semantic interpretation of visuo-spatial sensor data, e.g., coming from object tracking, eye tracking data, movement trajectories. The theory has been implemented within constraint logic programming to support integration into large scale AI projects. The theory is domain independent and has been applied in a range of domains, in which the capability to semantically interpret motion in visuo-spatial data is central. In this paper, we demonstrate its capabilities in the context of cognitive film studies for analysing visual perception of spectators by integrating the visual structure of a scene and spectators gaze acquired from eye tracking experiments.

Embodied visuo-locomotive experience analysis: immersive reality based summarisation of experiments in environment-behaviour studies

Evidence-based design (EBD) for architecture involves the study of post-occupancy behaviour of building users with the aim to provide an empirical basis for improving building performance [Hamilton and Watkins 2009]. Within EBD, the high-level, qualitative analysis of the embodied visuo-locomotive experience of representative groups of building users (e.g., children, senior citizens, individuals facing physical challenges) constitutes a foundational approach for understanding the impact of architectural design decisions, and functional building performance from the viewpoint of areas such as environmental psychology, wayfinding research, human visual perception studies, spatial cognition, and the built environment [Bhatt and Schultz 2016].

Semantic Analysis of (Reflectional) Visual Symmetry: A Human-Centred Computational Model for Declarative Explainability

We present a computational model for the semantic interpretation of symmetry in naturalistic scenes. Key features include a human-centred representation, and a declarative, explainable interpretation model supporting deep semantic question-answering founded on an integration of methods in knowledge representation and deep learning based computer vision. In the backdrop of the visual arts, we showcase the frameworks capability to generate human-centred, queryable, relational structures, also evaluating the framework with an empirical study on the human perception of visual symmetry. Our framework represents and is driven by the application of foundational, integrated Vision and Knowledge Representation and Reasoning methods for applications in the arts, and the psychological and social sciences.

Declarative Spatial Reasoning for Intelligent Cinematography

We present computational visuo-spatial representation and reasoning from the viewpoint of the research areas of artificial intelligence, spatial cognition and computation, and human-computer intera ...

Symmetry in the Eye of the Beholder

We study how subjective perception of symmetry can be computationally explained by features at different levels. We select 149 images with varying degrees of symmetry from photographs and movie frames and collect responses from 200 subjects. Each subject is shown 50 random images and asked to rate each image with one of four options: Not symmetric, Somewhat symmetric, Symmetric, and Highly symmetric. We measure the bilateral symmetry of an image by comparing CNN features across multiple levels between two vertical halves of an image. We use the AlexNet model pre-trained on the ImageNet dataset for extracting feature maps at all 5 convolutional layers. The extracted feature maps of the two bilateral halves are then compared to one another at different layers and spatial levels. The degree of similarity on different feature maps can then be used to model the range of symmetry an image can be seen to have. We train a multiclass SVM classifier to predict one of the four symmetry judgements based on these multi-level CNN symmetry scores. Our symmetry classifier has a very low accuracy when it needs to predict all observers’ responses equally well on individual images. However, our classification accuracies increase dramatically when each observer is modeled separately. Our results suggest that symmetry is in fact in the eye of the beholder: While some observers focus on high-level object semantics, others prefer low or mid level features in their symmetry assessment.