Thomas Uthmann

Learning competitive pricing strategies by multi-agent reinforcement learning

In electronic marketplaces automated and dynamic pricing is becoming increasingly popular. Agents that perform this task can improve themselves by learning from past observations, possibly using reinforcement learning techniques. Co-learning of several adaptive agents against each other may lead to unforeseen results and increasingly dynamic behavior of the market. In this article we shed some light on price developments arising from a simple price adaptation strategy. Furthermore, we examine several adaptive pricing strategies and their learning behavior in a co-learning scenario with different levels of competition. Q-learning manages to learn best-reply strategies well, but is expensive to train.

Automatic detection of ground glass opacities on lung HRCT using multiple neural networks.

The purpose of this study was to implement neural networks and expert rules for the automatic detection of ground glass opacities (GG) on high-resolution computed tomography (HRCT). Different approaches using self-organizing neural nets as well as classifications of lung HRCT with and without the use of explicit textural parameters have been applied in preliminary studies. In the present study a hybrid network of three single nets and an expert rule was applied for the detection of GG on 120 HRCT scans from 20 patients suffering from different lung diseases. Single nets alone were not capable to reliably detect or exclude GG since the false-positive rate was greater than 100 % with regard to the area truly involved, more than 50 pixels throughout, and the true-positive rate was greater than 95 %. The hybrid network correctly classified 91 of 120 scans. Mild GG was false positive in 15 cases with less than 50 pixels, which was judged not clinically relevant. The pitfalls were: partial volume effects of bronchovascular bundles and the chest wall. Motion artefacts and diaphragm were responsible for 11 misclassifications. Hybrid networks represent a promising tool for an automatic pathology-detecting system. They are ready to use as a diagnostic assistant for detection, quantification and follow-up of ground glass opacities, and further applications are underway.

Guest Editors' Introduction: Special Issue on Sensor Evolution

Artificial life researchers, in their attempts to create life-as-it-could-be, have widely studied both the behavior of animals and artifacts. Early precursors of life-like artificial systems such as Grey Walter’s tortoises [4] or Valentino Braitenberg’s vehicles [1] were already demonstrating that ALife research is strongly motivated by the desire to understand and create life-like behavior and (neural) control. Creating life-like behavior in simulations or robots has increased our understanding of the design and evolution of controllers for artificial systems. Despite the interrelationship between behavior, sensors, and other morphological characteristics of animal systems, the evolution of sensors is rarely the primary aim of scientific investigations. The choice of sensors for robots is often limited by practical or financial constraints, and sensors in simulations are often modeled without strong reference to biological sensors. In natural evolution one finds impressive examples of the principle of exploiting new sensory channels and the information they carry. Olfactory, tactile, auditory and visual, but also electrical and even magnetic senses have evolved in a multitude of variants, often utilizing organs not originally “intended” for the purpose they serve at present. Many biological sensors reach a degree of structural and functional complexity and of efficiency that is envied by engineers creating man-made sensors. Sensors enable animals to survive in dynamic and unstructured environments, to perceive and react appropriately to features in the biotic and abiotic environment, including members of their own species as well as predators and prey. The goal of synthesizing artificial sensors for hardware or software systems suggests a similar approach to that taken for generating life-like behavior, namely, using evolutionary techniques to explore design spaces and generate sensors that are specifically adapted with respect to environmental and other fitness-related constraints. Recent advancements in simulation as well as hardware technology provide increasing means to study sensor evolution [3]. The topic of sensor evolution is becoming a very modern and promising direction of research situated between biology, robotics, and artificial life. Research in this direction strives to provide

A study of the simulated evolution of the spectral sensitivity of visual agent receptors

In this article we study a model for the evolution of the spectral sensitivity of visual receptors for agents in a continuous virtual environment. The model uses a genetic algorithm (GA) to evolve the agent sensors along with the control of the agents by requiring the agents to solve certain tasks in the simulation environment. The properties of the evolved sensors are analyzed for different scenarios. In particular, it is shown that the GA is able to find a balance between sensor costs and agent performance in such a way that the spectral sensor sensitivity reflects the emission spectrum of the target objects and that the capability of the sensors to evolve can help the agents significantly in adapting to their task.

On a quantitative measure for modularity based on information theory

The concept of modularity appears to be crucial for many questions in the field of Artificial Life research. However, there have not been many quantitative measures for modularity that are both general and viable. In this paper we introduce a measure for modularity based on information theory. Due to the generality of the information theory formalism, this measure can be applied to various problems and models; some connections to other formalisms are presented.

Self-Organized Modularization in Evolutionary Algorithms

The principle of modularization has proven to be extremely successful in the field of technical applications and particularly for Software Engineering purposes. The question to be answered within the present article is whether mechanisms can also be identified within the framework of Evolutionary Computation that cause a modularization of solutions. We will concentrate on processes, where modularization results only from the typical evolutionary operators, i.e. selection and variation by recombination and mutation (and not, e.g., from special modularization operators). This is what we call Self-Organized Modularization. Based on a combination of two formalizations by Radcliffe and Altenberg, some quantitative measures of modularity are introduced. Particularly, we distinguish Built-in Modularityas an inherent property of a genotype and Effective Modularity, which depends on the rest of the population. These measures can easily be applied to a wide range of present Evolutionary Computation models. It will be shown, both theoretically and by simulation, that under certain conditions, Effective Modularity (as defined within this paper) can be a selection factor. This causes Self-Organized Modularization to take place. The experimental observations emphasize the importance of Effective Modularityin comparison with Built-in Modularity. Although the experimental results have been obtained using a minimalist toy model, they can lead to a number of consequences for existing models as well as for future approaches. Furthermore, the results suggest a complex self-amplification of highly modular equivalence classes in the case of respected relations. Since the well-known Holland schemata are just the equivalence classes of respected relations in most Simple Genetic Algorithms, this observation emphasizes the role of schemata as Building Blocks (in comparison with arbitrary subsets of the search space).

Quantification and Characterization of Pulmonary Emphysema in Multislice-CT

The new technology of the Multislice-CT provides volume data sets with approximately isotropic resolution, which permits a non invasive measurement of diffuse lung diseases like emphysema in the 3D space. The aim of our project is the development of a full automatic 3D CAD (Computer Aided Diagnosis) software tool for detection, quantification and characterization of emphysema in a thoracic CT data set. It should supply independently an analysis of an image data set to support the physician in clinical daily routine. In this paper we describe the developed 3D algorithms for the segmentation of the tracheo-bronchial tree, the lungs and the emphysema regions. We present different emphysema describing indices.

Evolution and Learning: Evolving Sensors in a Simple MDP Environment:

Natural intelligence and autonomous agents face difficulties when acting in information-dense environments. Assailed by a multitude of stimuli they have to make sense of the inflow of information, filtering and processing what is necessary, but discarding that which is unimportant. This paper aims at investigating the interactions between evolution of the sensorial channel extracting the information from the environment and the simultaneous individual adaptation of agent-control. Our particular goal is to study the influence of learning on the evolution of sensors, with learning duration being the tunable parameter. A genetic algorithm governs the evolution of sensors appropriate for the agent solving a simple grid world task. The performance of the agent is taken as fitness; ‘sensors’ are conceived as a map from environmental states to agent observations, and individual adaptation is modeled by Q-learning. Our experimental results show that due to the principles of cognitive economy learning and varying the degree thereof actually transforms the fitness landscape. In particular we identify a trade-off between learning speed (load) and sensor accuracy (error). These results are further reinforced by theoretical analysis: we derive an analytical measure for the quality of sensors based on the mutual entropy between the system of states and the selection of an optimal action, a concept recently proposed by Polani, Martinetz, and Kim.

Some Effects of Individual Learning on the Evolution of Sensors

In this paper, we present an abstract model of sensor evolution, where sensor development is only determined by artificial evolution and the adaptation of agent reactions is accomplished by individual learning. With the environment cast into a MDP framework, sensors can be conceived as a map from environmental states to agent observations and Reinforcement Learning algorithms can be utilised. On the basis of a simple gridworld scenario, we present some results of the interaction between individual learning and evolution of sensors.

Mainz Rolling Brains

Our agent team is the result of a development which had to take place under tight time limitations. The total development time available was slightly less than three months where over most of the time the team developers could invest no more than a few hours per week. The code was developed from scratch to improve over the design and quality of last year’s code. Thus one of the challenges was to keep a smooth development line and to avoid dead ends in the development, as well as to maintain a development environment in which a larger number of developers could work productively.