Ruohan Zhang

Fast and Precise Black and White Ball Detection for RoboCup Soccer

In 2016, UT Austin Villa claimed the Standard Platform League’s second place position at the RoboCup International Robot Soccer Competition in Leipzig, Germany as well as first place at both the RoboCup US Open in Brunswick, USA and the World RoboCup Conference in Beijing, China. This paper describes some of the key contributions that led to the team’s victories with a primary focus on our techniques for identifying and tracking black and white soccer balls. UT Austin Villa’s ball detection system was overhauled in order to transition from the league’s bright orange ball, used every year of the competition prior to 2016, to the truncated icosahedral pattern commonly associated with soccer balls.

UT Austin Villa: Project-Driven Research in AI and Robotics

UT Austin Villa is a robot soccer team that has competed in the annual RoboCup soccer competitions since 2003. The team has won several championships and has inspired research contributions spanning many topics in robotics and artificial intelligence. This article summarizes some of these research contributions and provides a snapshot into the current development status of the team. Educational uses of the teams code bases are also presented.

Model Checking for Safe Navigation Among Humans

We investigate the use of probabilistic model checking to synthesise optimal strategies for autonomous systems that operate among uncontrollable agents such as humans. To formally assess such uncontrollable behaviour, we use models obtained from reinforcement learning. These behaviour models are, e.g., based on data collected in experiments in which humans execute dynamic tasks in a virtual environment. We first describe a method to translate such behaviour models into Markov decision processes (MDPs). The composition of these MDPs with models for (controllable) autonomous systems gives rise to stochastic games (SGs). MDPs and SGs are amenable to probabilistic model checking which enables the synthesis of strategies that provably adhere to formal specifications such as probabilistic temporal logic constraints. Experiments with a prototype provide (1) systematic insights on the credibility and the characteristics of behavioural models and (2) methods for automated synthesis of strategies satisfying guarantees on their required characteristics in the presence of humans.

Cortical spike multiplexing using gamma frequency latencies

One of the fundamental problems in understanding the brain, in particular the cerebral cortex, is that we only have a partial understanding of the basic communication protocols that underlie signal transmission. This makes it difficult to interpret the significance of particular phenomena such as basic firing patterns and oscillations at different frequencies. There are, of course, useful models. Motivated by single-cell recording technology, Poisson statistics of cortical action potentials have long been a basic component in models of signal representation in the cortex. However, it is increasingly difficult to integrate Poisson spiking with spike timing signals in the gamma frequency spectrum. A potential way forward is being sparked by new technologies that allow the exploration of very low-level communication strategies. Specifically, the voltage potential of a cell’s soma now can be recorded with very high fidelity in vivo, allowing correlation of its fine structure to be correlated with behaviors. To interpret this data, we have developed a unified model (gamma spike multiplexing, or GSM) wherein a cell’s somatic gamma frequencies can modulate the generation of action potentials. Such spikes can be seen as the basis for a general-purpose method of modulating fast communication in cortical networks. In particular, the model has several important advantages over traditional formalisms: 1) It allows multiple, independent processes to run in parallel, greatly increasing the processing capability of the cortex 2) Its processing speed is 102 to 103 times faster than population coding methods 3) Its processes are not bound to specific locations, but migrate across cortical cells as a function of time, facilitating the maintenance of cortical cell calibration.

Modeling sensory-motor decisions in natural behavior

Although a standard reinforcement learning model can capture many aspects of reward-seeking behaviors, it may not be practical for modeling human natural behaviors because of the richness of dynamic environments and limitations in cognitive resources. We propose a modular reinforcement learning model that addresses these factors. Based on this model, a modular inverse reinforcement learning algorithm is developed to estimate both the rewards and discount factors from human behavioral data, which allows predictions of human navigation behaviors in virtual reality with high accuracy across different subjects and with different tasks. Complex human navigation trajectories in novel environments can be reproduced by an artificial agent that is based on the modular model. This model provides a strategy for estimating the subjective value of actions and how they influence sensory-motor decisions in natural behavior.