Xiaoyu Ge

Visual Detection of Unknown Objects in Video Games Using Qualitative Stability Analysis

Many current computer vision approaches for object detection can only detect objects that have been learned in advance. In this paper, we present a method that uses qualitative stability analysis to infer the existence of unknown objects in certain areas of the images based on gravity and stability of already detected objects. Our method recursively searches these areas for unknown objects until all detected objects form a stable structure or no new objects can be identified anymore. We evaluate our method using the popular video game Angry Birds. We only start with detecting the green pigs and are able to automatically identify and detect all essential game objects in all 400+ available levels. All objects can be accurately and reliably detected. Our method can be applied to other video games where objects obey gravity and are bound by polygons.

The Angry Birds AI Competition

The aim of the Angry Birds AI competition (AIBIRDS) is to build intelligent agents that can play new Angry Birds levels better than the best human players. This is surprisingly difficult for AI as it requires similar capabilities to what intelligent systems need for successfully interacting with the physical world, one of the grand challenges of AI. As such the competition offers a simplified and controlled environment for developing and testing the necessary AI technologies, a seamless integration of computer vision, machine learning, knowledge representation and reasoning, reasoning under uncertainty, planning, and heuristic search, among others. Over the past three years there have been significant improvements, but we are still a long way from reaching the ultimate aim and, thus, there are great opportunities for participants in this competition.

Tracking Perceptually Indistinguishable Objects Using Spatial Reasoning

Intelligent agents perceive the world mainly through images captured at different time points. Being able to track objects from one image to another is fundamental for understanding the changes of the world. Tracking becomes challenging when there are multiple perceptually indistinguishable objects (PIOs), i.e., objects that have the same appearance and cannot be visually distinguished. Then it is necessary to reidentify all PIOs whenever a new observation is made. In this paper we consider the case where changes of the world were caused by a single physical event and where matches between PIOs of subsequent observations must be consistent with the effects of the physical event.