Raymond D. Rimey

Control of Selective Perception using Bayes Nets and Decision Theory

A selective vision system sequentially collects evidence to answer a specific question with a desired level of confidence. Efficiency comes from processing the scene only where necessary, to the level of detail necessary, and with only the necessary operators. Knowledge representation and sequential decision making are central issues for selective vision, which takes advantage of prior knowledge of a domains abstract and geometrical structure (e.g., “part-of” and “adjacent” relationships), and also uses information from a scene instance gathered during analysis. The TEA-1 selective vision system uses Bayes nets for representation, benefit-cost analysis for control of visual and nonvisual actions; and its data structures and decision-making algorithms provide a general, reusable framework. TEA-1 solves the T-world problem, an abstraction of a large set of scene domains and tasks. Some factors that affect the success of selective perception are analyzed by using TEA-1 to solve ensembles of randomly produced, simulated T-world problems. Experimental results with a real-world T-world problem, dinner table scenes, are also presented.

Controlling eye movements with hidden Markov models

Advances in technology and in active vision research allow and encourage sequential visual information acquisition. Hidden Markov models (HMMs) can represent probabilistic sequences and probabilistic graph structures: here we explore their use in controlling the acquisition of visual information. We include a brief tutorial with two examples: (1) use input sequences to derive an aspect graph and (2) similarly derive a finite state machine for control of visual processing.The first main topic is the use of HMMs in both their learning and generative modes, and their augmentation to allow inputs sensed during generation to modify the generated outputs temporarily or permanently. We propose these augmented HMMs as a theory of adaptive skill acquisition and generation. The second main topic builds on the first: the augmented HMMs can be used for knowledge fusion. We give an example, the what-where-AHMM, which creates a hybrid skill from separate skills based on object location and object identity. Insofar as low-level skills can be learned from the output of high-level cognitive processes, AHMMs can provide a link between high-level and low-level vision.

Where to Look Next Using a Bayes Net: Incorporating Geometric Relations

A task-oriented system is one that performs the minimum effort necessary to solve a specified task. Depending on the task, the system decides which information to gather, which operators to use at which resolution, and where to apply them. We have been developing the basic framework of a task-oriented computer vision system, called TEA, that uses Bayes nets and a maximum expected utility decision rule. In this paper we present a method for incorporating geometric relations into a Bayes net, and then show how relational knowledge and evidence enables a task-oriented system to restrict visual processing to particular areas of a scene by making camera movements and by only processing a portion of the data in an image.

Task-specific utility in a general Bayes net vision system

TEA is a task-oriented computer vision system that uses Bayes nets and a maximum expected-utility decision rule to choose a sequence of task-dependent and opportunistic visual operations on the basis of their cost and (present and future) benefit. The authors discuss technical problems regarding utilities, present TEA-1s utility function (which approximates a two-step lookahead), and compare it to various simpler utility functions in experiments with real and simulated scenes.<<ETX>>

Sequences, structure, and active vision

Sequences of symbols generated by a visual and action sequence provide information about the natural structure of the world. HMMs (hidden Markov models) provide one way to learn (recover), store, produce, manipulate, and analyze both visual sequences and associated knowledge structures for computer vision.<<ETX>>