Benjamin Kuipers

Modeling Spatial Knowledge

A persons cognitive map, or knowledge of large-scale space, is built up from observations gathered as he travels through the environment. It acts as a problem solver to find routes and relative positions, as well as describing the current location. The TOUR model captures the multiple representations that make up the cognitive map, the problem-solving strategies it uses, and the mechanisms for assimilating new information. The representations have rich collections of states of partial knowledge, which support many of the performance characteristics of common-sense knowledge.

The Spatial Semantic Hierarchy

Abstract The Spatial Semantic Hierarchy is a model of knowledge of large-scale space consisting of multiple interacting representations, both qualitative and quantitative. The SSH is inspired by the properties of the human cognitive map, and is intended to serve both as a model of the human cognitive map and as a method for robot exploration and map-building. The multiple levels of the SSH express states of partial knowledge, and thus enable the human or robotic agent to deal robustly with uncertainty during both learning and problem-solving. The control level represents useful patterns of sensorimotor interaction with the world in the form of trajectory-following and hill-climbing control laws leading to locally distinctive states. Local geometric maps in local frames of reference can be constructed at the control level to serve as observers for control laws in particular neighborhoods. The causal level abstracts continuous behavior among distinctive states into a discrete model consisting of states linked by actions. The topological level introduces the external ontology of places, paths and regions by abduction to explain the observed pattern of states and actions at the causal level. Quantitative knowledge at the control, causal and topological levels supports a “patchwork map” of local geometric frames of reference linked by causal and topological connections. The patchwork map can be merged into a single global frame of reference at the metrical level when sufficient information and computational resources are available. We describe the assumptions and guarantees behind the generality of the SSH across environments and sensorimotor systems. Evidence is presented from several partial implementations of the SSH on simulated and physical robots.

Commonsense reasoning about causality: deriving behavior from structure

Abstract This paper presents a qualitative-reasoning method for predicting the behavior of mechanisms characterized by continuous, time-varying parameters. The structure of a mechanism is described in terms of a set of parameters and the constraints that hold among them: essentially a ‘qualitative differential equation’. The qualitative-behavior description consists of a discrete set of time-points, at which the values of the parameters are described in terms of ordinal relations and directions of change. The behavioral description, or envisionment, is derived by two sets of rules: propagation rules which elaborate the description of the current time-point, and prediction rules which determine what is known about the next qualitatively distinct state of the mechanism. A detailed example shows how the envisionment method can detect a previously unsuspected landmark point at which the system is in stable equilibrium.

Recognizing human actions by attributes

In this paper we explore the idea of using high-level semantic concepts, also called attributes, to represent human actions from videos and argue that attributes enable the construction of more descriptive models for human action recognition. We propose a unified framework wherein manually specified attributes are: i) selected in a discriminative fashion so as to account for intra-class variability; ii) coherently integrated with data-driven attributes to make the attribute set more descriptive. Data-driven attributes are automatically inferred from the training data using an information theoretic approach. Our framework is built upon a latent SVM formulation where latent variables capture the degree of importance of each attribute for each action class. We also demonstrate that our attribute-based action representation can be effectively used to design a recognition procedure for classifying novel action classes for which no training samples are available. We test our approach on several publicly available datasets and obtain promising results that quantitatively demonstrate our theoretical claims.

A Description of Think Aloud Method and Protocol Analysis

Think Aloud (TA) studies provide rich verbal data about reasoning during a problem solving task. Using TA and protocol analysis, investigators can identify the information that is concentrated on during problem solving and how that information is used to facilitate problem resolution. From this, inferences can be made about the reasoning processes that were used during the problem-solving task. In the past, the validity of data obtained from TA studies has been suspect because of inconsistencies in data collection and the inability to verify findings obtained from the slow, laborious process of protocol analysis. This article describes a means of obtaining more accurate verbal data and analyzing it in a standardized step-by-step manner.

Causal reasoning in medicine: Analysis of a protocol

The ability to identify and represent the knowledge that a human expert has about a particular domain is a key method in the creation of an expert computer system. The first part of this paper demonstrates a methodology for collecting and analyzing observations of experts at work, in order to find the conceptual framework used for the particular domain. The second part develops a representation for qualitative knowledge of the structure and behavior of a mechanism. The qualitative simulation, or envisionment, process is given a qualitative structural description of a mechanism and some initialization information, and produces a detailed description of the mechanisms behavior. The simulation process has been fully implemented, and its results are shown for a particular disease mechanisms in nephrology. This vertical slice of the construction of a cognitive model demonstrates an effective knowledge acquisition method for the purpose of determining the structure of the representation itself, not simply the content of the knowledge to be encoded in that representation. Most importantly, it demonstrates the interaction among constraints derived from the textbook knowledge of the domain, from observations of the human expert, and from the computational requirements of successful performance.

Cross-view action recognition via view knowledge transfer

In this paper, we present a novel approach to recognizing human actions from different views by view knowledge transfer. An action is originally modelled as a bag of visual-words (BoVW), which is sensitive to view changes. We argue that, as opposed to visual words, there exist some higher level features which can be shared across views and enable the connection of action models for different views. To discover these features, we use a bipartite graph to model two view-dependent vocabularies, then apply bipartite graph partitioning to co-cluster two vocabularies into visual-word clusters called bilingual-words (i.e., high-level features), which can bridge the semantic gap across view-dependent vocabularies. Consequently, we can transfer a BoVW action model into a bag-of-bilingual-words (BoBW) model, which is more discriminative in the presence of view changes. We tested our approach on the IXMAS data set and obtained very promising results. Moreover, to further fuse view knowledge from multiple views, we apply a Locally Weighted Ensemble scheme to dynamically weight transferred models based on the local distribution structure around each test example. This process can further improve the average recognition rate by about 7%.

Local metrical and global topological maps in the hybrid spatial semantic hierarchy

Topological and metrical methods for representing spatial knowledge have complementary strengths. We present a hybrid extension to the spatial semantic hierarchy that combines their strengths and avoids their weaknesses. Metrical SLAM methods are used to build local maps of small-scale space within the sensory horizon of the agent, while topological methods are used to represent the structure of large-scale space. We describe how a local perceptual map is analyzed to identify a local topology description and is abstracted to a topological place. The map building method creates a set of topological map hypotheses that are consistent with travel experience. The set of maps is guaranteed under reasonable assumptions to include the correct map. We demonstrate the method on a real environment with multiple nested large-scale loops.

Towards a general theory of topological maps

We present a general theory of topological maps whereby sensory input, topological and local metrical information are combined to define the topological maps explaining such information. Topological maps correspond to the minimal models of an axiomatic theory describing the relationships between the different sources of information explained by a map. We use a circumscriptive theory to specify the minimal models associated with this representation.The theory here proposed is independent of the exploration strategy the agent follows when building a map. We provide an algorithm to calculate the models of the theory. This algorithm supports different exploration strategies and facilitates map disambiguation when perceptual aliasing arises.

Qualitative reasoning: modeling and simulation with incomplete knowledge

Abstract Recently developed methods for qualitative reasoning may fill an important gap in the modeling and control toolkit. Qualitative reasoning methods provide greater expressive power for states of incomplete knowledge than differential or difference equations, and thus make it possible to build models without incorporating assumptions of linearity or specific values for incompletely known constants. Even with incomplete knowledge, there is enough information in a qualitative description to support qualitative simulation, predicting the possible behaviors of an incompletely described system. We survey results from several approaches to qualitative reasoning, and provide a detailed example of the application of these methods to a simple problem. The mathematical validity of qualitative simulation is also assessed. Initial results have been encouraging, and steps are now being taken to develop additional mathematical power, hierarchical decomposition methods, and incremental quantitative constraints, to make qualitative reasoning into a formal reasoning method useful on realistic problems.