Paulo E. Santos

Protocols from perceptual observations

This paper presents a cognitive vision system capable of autonomously learning protocols from perceptual observations of dynamic scenes. The work is motivated by the aim of creating a synthetic agent that can observe a scene containing interactions between unknown objects and agents, and learn models of these sufficient to act in accordance with the implicit protocols present in the scene. Discrete concepts (utterances and object properties), and temporal protocols involving these concepts, are learned in an unsupervised manner from continuous sensor input alone. Crucial to this learning process are methods for spatio-temporal attention applied to the audio and visual sensor data. These identify subsets of the sensor data relating to discrete concepts. Clustering within continuous feature spaces is used to learn object property and utterance models from processed sensor data, forming a symbolic description. The progol Inductive Logic Programming system is subsequently used to learn symbolic models of the temporal protocols presented in the presence of noise and over-representation in the symbolic data input to it. The models learned are used to drive a synthetic agent that can interact with the world in a semi-natural way. The system has been evaluated in the domain of table-top game playing and has been shown to be successful at learning protocol behaviours in such real-world audio-visual environments.

PID, LQR and LQR-PID on a quadcopter platform

This paper aims to present a comparison between different controllers to be used in a dynamic model of a quadcopter platform. The controllers assumed in this work are an ITAE tuned PID, a classic LQR controller and a PID tuned with a LQR loop. The results were obtained through simulations for 10 different attitudes of the quadcopter, however, in this paper simulation results will be presented for the vertical attitude only (the remainder are analogous and were omitted for brevity).

The perception and content of cast shadows: an interdisciplinary review

Abstract Recently, psychologists have turned their attention to the study of cast shadows and demonstrated that the human perceptual system values information from shadows very highly in the perception of spatial qualities, sometimes to the detriment of other cues. However with some notable and recent exceptions, computer vision systems treat cast shadows not as signal but as noise. This paper provides a concise yet comprehensive review of the literature on cast shadow perception from across the cognitive sciences, including the theoretical information available, the perception of shadows in human and machine vision, and the ways in which shadows can be used.

Cognitive Vision: Integrating Symbolic Qualitative Representations with Computer Vision

We describe the challenge of combining continuous computer vision techniques and qualitative, symbolic methods to achieve a system capable of cognitive vision. Key to a truly cognitive system, is the ability to learn: to be able to build and use models constructed autonomously from sensory input. In this paper we overview a number of steps we have taken along the route to the construction of such a system, and discuss some remaining challenges.

Formalising the Fisherman's Folly puzzle

This paper investigates the challenging problem of encoding the common sense knowledge involved in the manipulation of spatial objects from a reasoning about actions and change perspective. In particular, we propose a formal solution to a puzzle composed of non-trivial objects (such as holes and strings) assuming a version of the Situation Calculus written over first-order Equilibrium Logic, whose models generalise the stable model semantics.

Reasoning about Depth and Motion from an Observer's Viewpoint

ABSTRACT The goal of this paper is to present a logic-based formalism for representing knowledge about objects in space and their movements, and show how this knowledge could be built up from the viewpoint of an observer immersed in a dynamic world. In this paper space is represented using functions that extract attributes of depth, size and distance from snapshots of the world. These attributes compose a novel spatial reasoning system named Depth Profile Calculus (DPC). Transitions between qualitative relations involving these attributes are represented by an extension of this calculus called Dynamic Depth Profile Calculus (DDPC). We argue that knowledge about objects in the world could be built up via a process of abduction on DDPC relations.

Transferring knowledge as heuristics in reinforcement learning

The goal of this paper is to propose and analyse a transfer learning meta-algorithm that allows the implementation of distinct methods using heuristics to accelerate a Reinforcement Learning procedure in one domain (the target) that are obtained from another (simpler) domain (the source domain). This meta-algorithm works in three stages: first, it uses a Reinforcement Learning step to learn a task on the source domain, storing the knowledge thus obtained in a case base; second, it does an unsupervised mapping of the source-domain actions to the target-domain actions; and, third, the case base obtained in the first stage is used as heuristics to speed up the learning process in the target domain.A set of empirical evaluations were conducted in two target domains: the 3D mountain car (using a learned case base from a 2D simulation) and stability learning for a humanoid robot in the Robocup 3D Soccer Simulator (that uses knowledge learned from the Acrobot domain). The results attest that our transfer learning algorithm outperforms recent heuristically-accelerated reinforcement learning and transfer learning algorithms.

Qualitative robot localisation using information from cast shadows

Recently, cognitive psychologists and others have turned their attention to the formerly neglected study of shadows, and the information they purvey. These studies show that the human perceptual system values information from shadows very highly, particularly in the perception of depth, even to the detriment of other cues. However with a few notable exceptions, computer vision systems have treated shadows not as signal but as noise. This paper makes a step towards redressing this imbalance by considering the formal representation of shadows. We take one particular aspect of reasoning about shadows, developing the idea that shadows carry information about a fragment of the viewpoint of the light source. We start from the observation that the region on which the shadow is cast is occluded by the caster with respect to the light source and build a qualitative theory about shadows using a region-based spatial formalism about occlusion. Using this spatial formalism and a machine vision system we are able to draw simple conclusions about domain objects and egolocation for a mobile robot.

Perceptual Narratives of Space and Motion for Semantic Interpretation of Visual Data

We propose a commonsense theory of space and motion for the high-level semantic interpretation of dynamic scenes. The theory provides primitives for commonsense representation and reasoning with qualitative spatial relations, depth profiles, and spatio-temporal change; these may be combined with probabilistic methods for modelling and hypothesising event and object relations. The proposed framework has been implemented as a general activity abstraction and reasoning engine, which we demonstrate by generating declaratively grounded visuo-spatial narratives of perceptual input from vision and depth sensors for a benchmark scenario.

Predictive and descriptive approaches to learning game rules from vision data

Systems able to learn from visual observations have a great deal of potential for autonomous robotics, scientific discovery, and many other fields as the necessity to generalise from visual observation (from a quotidian scene or from the results of a scientific enquiry) is inherent in various domains. We describe an application to learning rules of a dice game using data from a vision system observing the game being played. In this paper, we experimented with two broad approaches: (i) a predictive learning approach with the Progol system, where explicit concept learning problems are posed and solved, and (ii) a descriptive learning approach with the HR system, where a general theory is formed with no specific problem solving task in mind and rules are extracted from the theory.