Christoph Salge

Bivariate Measure of Redundant Information

We define a measure of redundant information based on projections in the space of probability distributions. Redundant information between random variables is information that is shared between those variables. But, in contrast to mutual information, redundant information denotes information that is shared about the outcome of a third variable. Formalizing this concept, and being able to measure it, is required for the non-negative decomposition of mutual information into redundant and synergistic information. Previous attempts to formalize redundant or synergistic information struggle to capture some desired properties. We introduce a new formalism for redundant information and prove that it satisfies all the properties necessary outlined in earlier work, as well as an additional criterion that we propose to be necessary to capture redundancy. We also demonstrate the behavior of this new measure for several examples, compare it to previous measures, and apply it to the decomposition of transfer entropy.

Empowerment -- an Introduction

Is it better for you to own a corkscrew or not? If asked, you as a human being would likely say “yes”, but more importantly, you are somehow able to make this decision. You are able to decide this, even if your current acute problems or task do not include opening a wine bottle. Similarly, it is also unlikely that you evaluated several possible trajectories your life could take and looked at them with and without a corkscrew, and then measured your survival or reproductive fitness in each. When you, as a human cognitive agent, made this decision, you were likely relying on a behavioural “proxy”, an internal motivation that abstracts the problem of evaluating a decision impact on your overall life, but evaluating it in regard to some simple fitness function. One example would be the idea of curiosity, urging you to act so that your experience new sensations and learn about the environment. On average, this should lead to better and richer models of the world, which give you a better chance of reaching your ultimate goals of survival and reproduction.

APPROXIMATION OF EMPOWERMENT IN THE CONTINUOUS DOMAIN

The empowerment formalism offers a goal-independent utility function fully derived from an agents embodiment. It produces intrinsic motivations which can be used to generate self-organizing behaviors in agents. One obstacle to the application of empowerment in more demanding (esp. continuous) domains is that previous ways of calculating empowerment have been very time consuming and only provided a proof-of-concept. In this paper we present a new approach to efficiently approximate empowerment as a parallel, linear, Gaussian channel capacity problem. We use pendulum balancing to demonstrate this new method, and compare it to earlier approximation methods.

Changing the Environment Based on Empowerment as Intrinsic Motivation

Christoph Salge *, Cornelius Glackin and Daniel PolaniAdaptive Systems Research Group, University of Hertfordshire, College Lane,Hatfield AL10 9AB, UK; E-Mails: c.salge@herts.ac.uk; c.glackin2@herts.ac.uk; d.polani@herts.ac.uk* Author to whom correspondence should be addressed; E-Mail: c.salge@herts.ac.uk;Tel.:+44-1707-284490.Received: 28 February 2014; in revised form: 28 April 2014 / Accepted: 4 May 2014 /Published: 21 May 2014Abstract: One aspect of intelligence is the ability to restructure your own environment sothat the world you live in becomes more beneficial to you. In this paper we investigate howthe information-theoretic measure of agent empowerment can provide a task-independent,intrinsic motivation to restructure the world. We show how changes in embodiment andin the environment change the resulting behaviour of the agent and the artefacts left in theworld. For this purpose, we introduce an approximation of the established empowermentformalism based on sparse sampling, which is simpler and significantly faster to computefor deterministic dynamics. Sparse sampling also introduces a degree of randomness into thedecision making process, which turns out to beneficial for some cases. We then utilize themeasure to generate agent behaviour for different agent embodiments in a Minecraft-inspiredthree dimensional block world. The paradigmatic results demonstrate that empowerment canbe used as a suitable generic intrinsic motivation to not only generate actions in given staticenvironments, as shown in the past, but also to modify existing environmental conditions.In doing so, the emerging strategies to modify an agent’s environment turn out to bemeaningful to the specific agent capabilities, i.e., de facto to its embodiment.Keywords: empowerment; intrinsic motivation; information theory

Using genetically optimized artificial intelligence to improve gameplaying fun for strategical games

Fun in computer games depends on many factors. While some factors like uniqueness and humor can only be measured by human subjects, in a strategical game, the rule system is an important and measurable factor. Classics like chess and GO have a millennia-old story of success, based on clever rule design. They only have a few rules, are relatively easy to understand, but still they have myriads of possibilities. Testing the deepness of a rule-set is very hard, especially for a rule system as complex as in a classic strategic computer game. It is necessary, though, to ensure prolonged gaming fun. In our approach, we use artificial intelligence (AI) to simulate hours of beta-testing the given rules, tweaking the rules to provide more game-playing fun and deepness. To avoid making the AI a mirror of its programmers gaming preferences, we not only evolved the AI with a genetic algorithm, but also used three fundamentally different AI paradigms to find boring loopholes, inefficient game mechanisms and, last but not least, complex erroneous behavior.

CORBYS cognitive control architecture for robotic follower

In this paper the novel generic cognitive robot control architecture CORBYS is presented. The objective of the CORBYS architecture is the integration of high-level cognitive modules to support robot functioning in dynamic environments including interacting with humans. This paper presents the preliminary integration of the CORBYS architecture to support a robotic follower. Experimental results on high-level empowerment-based trajectory planning have demonstrated the effectiveness of ROS-based communication between distributed modules developed in a multi-site research environment as typical for distributed collaborative projects such as CORBYS.

Relevant Information as a formalised approach to evaluate game mechanics

We present a new approach to use adaptive AI and Information Theory to aid the evaluation of game mechanics. Being able to evaluate the core game mechanics early during production is useful to improve the quality of a game, and ultimately, player satisfaction. A current problem with automated game evaluation via AI is to define measurable parameters that correlate to the quality of the game mechanics. We apply the Information Theory based concept of “Relevant Information” to this problem and argue that there is a relation between enjoyment related game-play properties and Relevant Information for an AI playing the game. We also demonstrate, with a simple game implementation, a.) how an adaptive AI can be used to approximate the Relevant Information, b.) how those measurable numerical values relate to certain game design flaws c.) how this knowledge can be used to improve the game.

Zipf's Law: Balancing Signal Usage Cost and Communication Efficiency.

We propose a model that explains the reliable emergence of power laws (e.g., Zipf’s law) during the development of different human languages. The model incorporates the principle of least effort in communications, minimizing a combination of the information-theoretic communication inefficiency and direct signal cost. We prove a general relationship, for all optimal languages, between the signal cost distribution and the resulting distribution of signals. Zipf’s law then emerges for logarithmic signal cost distributions, which is the cost distribution expected for words constructed from letters or phonemes.

Intrinsically motivated general companion NPCs via Coupled Empowerment Maximisation

Non-player characters (NPCs) in games are traditionally hard-coded or dependent on pre-specified goals, and consequently struggle to behave sensibly in ever-changing and possibly unpredictable game worlds. To make them fit for new developments in procedural content generation, we introduce the principle of Coupled Empowerment Maximisation as an intrinsic motivation for game NPCs. We focus on the development of a general game companion, designed to support the player in achieving their goals. We evaluate our approach against three intuitive and abstract companion duties. We develop dedicated scenarios for each duty in a dungeon-crawler game testbed, and provide qualitative evidence that the emergent NPC behaviour fulfils these duties. We argue that this generic approach can speed up NPC AI development, improve automatic game evolution and introduce NPCs to full game-generation systems.

Learning gait by therapist demonstration for natural-like walking with the CORBYS powered orthosis

The number of mechanical degrees of freedom (DoFs) within rehabilitation robots directly influences the scope of the movements that a subject can perform when training walking. Currently, gait rehabilitation robots have a limited number of mechanical DoFs, as a consequence this limits the movements these robots can make possible. In this paper, the novel gait rehabilitation system CORBYS is presented which consists of the mobile platform and a powered orthosis which is attached to the platform. The CORBYS powered orthosis has 16 DoFs enabling more physiological movements, making it a state-of-the-art gait rehabilitation robotic system. With the sufficient number of DoFs to enable natural-like walking, the CORBYS robotic system enables the integration of the “learning gait by therapist demonstration” paradigm. This paper presents the fully integrated functional CORBYS gait rehabilitation system, with the focus on the implementation aspects which enable generation of the reference gait trajectory through learning by therapist demonstration, and the use of the generated trajectory in the robotic therapy session. The results of the initial evaluation of the robotic system obtained in tests with a selected patient are given in the paper.