Damian M. Lyons

A background model initialization algorithm for video surveillance

Many motion detection and tracking algorithms rely on the process of background subtraction, a technique which detects changes from a model of the background scene. We present a new algorithm for the purpose of background model initialization. The algorithm takes as input a video sequence in which moving objects are present, and outputs a statistical background model describing the static parts of the scene. Multiple hypotheses of the background value at each pixel are generated by locating periods of stable intensity in the sequence. The likelihood of each hypothesis is then evaluated using optical flow information from the neighborhood around the pixel, and the most likely hypothesis is chosen to represent the background. Our results are compared with those of several standard background modeling techniques using surveillance video of humans in indoor environments.

A formal model of computation for sensory-based robotics

It is noted that almost all attempts to construct special-purpose robot programming languages have proceeded by taking a computer-programming language and adding some special primitives. Here, the authors have taken the approach of trying to define computation at its most primitive level in terms of the characteristics of the robot domain. They construct a special model of computation, called RS (robot schemas), with properties designed to facilitate sensory-based robot programming. This approach offers the potential to construct robot task representations which are easy to use and concise, and which execute in an efficient manner. The authors define the model formally using port automata. These definitions ensure consistency and well-definedness, and the facilitate plan verification and automatic plan generation. >

Representing and analyzing action plans as networks of concurrent processes

The problem of constructing a plan representation that can deal with the complexity of representing and analyzing robot behavior in uncertain and dynamic environments is addressed. A concurrent-process based representation is developed which represents both the plan (or controller) and the uncertain and dynamic environment in which the plan operates. A methodology is outlined for analyzing the behavior of this interacting system of plan and world. This methodology is illustrated with a mixed-batch example from the domain of robotic kitting. To balance the theoretical work, a description of the implemented robot kitting cell is presented. >

Assembly and task planning: a taxonomy

After over two decades of research in the field of assembly and task planning, an IEEE Technical Committee on Assembly and Task Planning has been created. One of the first activities of this committee has been to develop a taxonomy of the field. We will now present this taxonomy, which should be viewed as a dynamic description of the field that will change shape as the field progresses. The reasons for developing such a taxonomy are many: to aid in building a coherent picture of the field which is particularly useful to newcomers to assembly and task planning, to help active researchers better understand how their work relates to the rest of the field. Finally such a taxonomy is useful for administrative purposes such as aiding program committee members and editors in selecting reviewers and as an aid in selecting keywords to be associated with various articles. To further aid the reader of this taxonomy, we have incorporated into it an extensive, but by no means complete, bibliography.<<ETX>>

Combining multiple scoring systems for target tracking using rank-score characteristics

Video target tracking is the process of estimating the current state, and predicting the future state of a target from a sequence of video sensor measurements. Multitarget video tracking is complicated by the fact that targets can occlude one another, affecting video feature measurements in a highly non-linear and difficult to model fashion. In this paper, we apply a multisensory fusion approach to the problem of multitarget video tracking with occlusion. The approach is based on a data-driven method (CFA) to selecting the features and fusion operations that improve a performance criterion. Each sensory cue is treated as a scoring system. Scoring behavior is characterized by a rank-score function. A diversity measure, based on the variation in rank-score functions, is used to dynamically select the scoring systems and fusion operations that produce the best tracking performance. The relationship between the diversity measure and the tracking accuracy of two fusion operations, a linear score combination and an average rank combination, is evaluated on a set of 12 video sequences. These results demonstrate that using the rank-score characteristic as a diversity measure is an effective method to dynamically select scoring systems and fusion operations that improve the performance of multitarget video tracking with occlusions.

Planning for reactive robot behavior

A robot system operating in an environment in which there is uncertainty and change needs to combine the ability to react with the ability to plan ahead. The authors had previously proposed a solution to the problems of integrating planning and reaction: cast planning as adaptation of a reactive system. They extend the theoretical treatment of the planner, including the effect of its iterated adaptation on the reactor, and describe the current implementation environment and a working example.<<ETX>>

A practical approach to integrating reaction and deliberation

Abstract A robot system operating in an environment in which there is uncertainty and change needs to integrate the ability to react with the ability to plan ahead. In this paper, we present a novel approach to this integration for a practical robot problem, the kitting robot. In our approach, planning is cast as the ongoing adaptation of a reactive system, incrementally bringing its behavior into line with a set of goals. A method is presented whereby the planner can make iterative improvements to the reactor so that it eventually converges on the ideal reactor (a concept related to the universal plan) for its environment and set of planner goals. To balance this theoretical work, our current implementation on a Puma 560 robot is overviewed.

Planning as incremental adaptation of a reactive system

Abstract The importance of solving the problem of integrating deliberative (“planning”) capabilities and reactive capabilities when building robust, ‘real-world’ robot systems is becoming widely accepted (Bresina and Drummond, 1990; Fraichard and Laugier, 1991; McDermott, 1991). This paper presents a solution to this problem: cast planning as the incremental adaptation of a reactive system to suit changes in goals or the environment. Our application domain is a manufacturing problem - robotic kitting. This paper represents an advance on existing work in two ways: It presents and formally examines an architecture that incorporates the benefits of a deliberative component without compromising the reactive component. Secondly, it provides the first set of performance statistics in the literature for this class of system. In our approach, the reactive system (the reactor) is a real-time system that continually interacts with the environment, and the planner is a separate and concurrent system that incrementally ‘tunes’ the behavior of the reactor to ensure that goals are achieved. We call this the planner-reactor approach. The reactor is described using a formal framework for representing flexible robot plans, the RS model (Lyons, 1990; Lyons and Arbib, 1989). Thus, the behavior of the reactor, and the rules by which the reactor can be modified, become open to mathematical analysis. We employ this to determine the constraints the planner must abide by to make safe adaptations and to ensure that incremental adaptations converge to a desired reactor. We discuss our current implementation of planner and reactor, work through an example from the kitting robot application, and present implementation results.

Achieving robustness by casting planning as adaptation of a reactive system

Classical artificial intelligence planning is not sufficiently robust in uncertain and dynamic environments. Reactive approaches are robust in some environments-namely those for which they have been programmed. An approach that integrates a priori planning with reaction to increase robustness is presented. As motivation, a practical robot problem, the kitting robot, is presented. Solving this problem demands a system that can make timely and robust actions in an uncertain environment. A solution to the kitting robot problem in which planning is cast as adaptation of a reactive system to suit changes in the goals or environment is outlined. The reactive system (the reactor) is based on a formal model for representing flexible robot plans, the RS model. Thus, it was possible to formalize the mechanisms by which the planner improves the behavior of the reactor. This system was implemented to control a Puma-560 robot equipped with visual sensing.<<ETX>>

Visual Surveillance in Retail Stores and in the Home

This paper presents an overview of video surveillance activities at Philips Research USA, which serves as a research arm of Philips Communications, Security and Imaging, a global leader in the design and manufacture of advanced automatic security systems. We concentrate on two application domains: professional security market, with emphasis on retail monitoring, and a low-cost, automated residential security. The main application for pan-tilt-zoom (PTZ) cameras is typically tracking of intruders throughout the facility. In addition, camera calibration is an important capability, because it allows geometric reasoning with respect to the floor plan as well as enhanced control of the camera. Another major component of our work concerns video content analysis, the detection of security related objects and events from video. The system typically processes video in real-time and can provide immediate alarms to alert the security operator. Relevant information is also stored in a database so that it can be efficiently retrieved later. The third and final topic discussed in the paper is a residential monitoring application, namely an intruder detection system. We describe detection of moving objects robust to changes in lighting and an object classification scheme based on radial-basis networks.