Teresa Ko

Background Subtraction on Distributions

Environmental monitoring applications present a challenge to current background subtraction algorithms that analyze the temporal variability of pixel intensities, due to the complex texture and motion of the scene. They also present a challenge to segmentation algorithms that compare intensity or color distributions between the foreground and the background in each image independently, because objects of interest such as animals have adapted to blend in. Therefore, we have developed a background modeling and subtraction scheme that analyzes the temporal variation of intensity or color distributions, instead of either looking at temporal variation of point statistics, or the spatial variation of region statistics in isolation. Distributional signatures are less sensitive to movements of the textured background, and at the same time they are more robust than individual pixel statistics in detecting foreground objects. They also enable slow background update, which is crucial in monitoring applications where processing power comes at a premium, and where foreground objects, when present, may move less than the background and therefore disappear into it when a fast update scheme is used. Our approach compares favorably with the state of the art both in generic low-level detection metrics, as well as in application-dependent criteria.

Heartbeat of a nest: Using imagers as biological sensors

We present a scalable end-to-end system for vision-based monitoring of natural environments, and illustrate its use for the analysis of avian nesting cycles. Our system enables automated analysis of thousands of images, where manual processing would be infeasible. We automate the analysis of raw imaging data using statistics that are tailored to the task of interest. These “features” are a representation to be fed to classifiers that exploit spatial and temporal consistencies. Our testbed can detect the presence or absence of a bird with an accuracy of 82%, count eggs with an accuracy of 84%, and detect the inception of the nesting stage within a day. Our results demonstrate the challenges and potential benefits of using imagers as biological sensors. An exploration of system performance under varying image resolution and frame rate suggest that an in situ adaptive vision system is technically feasible.

Warping background subtraction

We present a background model that differentiates between background motion and foreground objects. Unlike most models that represent the variability of pixel intensity at a particular location in the image, we model the underlying warping of pixel locations arising from background motion. The background is modeled as a set of warping layers, where at any given time, different layers may be visible due to the motion of an occluding layer. Foreground regions are thus defined as those that cannot be modeled by some composition of some warping of these background layers. We illustrate this concept by first reducing the possible warps to those where the pixels are restricted to displacements within a spatial neighborhood, and then learning the appropriate size of that spatial neighborhood. Then, we show how changes in intensity/color histograms of pixel neighborhoods can be used to discriminate foreground and background regions. We find that this approach compares favorably with the state of the art, while requiring less computation.

Exploring Tradeoffs in Accuracy, Energy and Latency of Scale Invariant Feature Transform in Wireless Camera Networks

Advances in DSP technology create important avenues of research for embedded vision. One such avenue is the investigation of tradeoffs amongst system parameters which affect the energy, accuracy, and latency of the overall system. This paper reports work on benchmarking the performance and cost of scale invariant feature transform (SIFT) for visual classification on a Blackfin DSP processor. Through measurements and modeling of the camera sensor node, we investigate system performance (classification accuracy, latency, energy consumption) in light of image resolution, arithmetic precision, location of processing (local vs. server-side), and processor speed. A case study on counting eggs during avian nesting season is used to experimentally determine the tradeoffs of different design parameters and discuss implications to other application domains.

Embedded Imagers: Detecting, Localizing, and Recognizing Objects and Events in Natural Habitats

Imaging sensors, or “imagers,” embedded in the natural environment enable remote collection of large quantities of data, thus easing the design and deployment of sensing systems in a variety of application domains. Yet, the data collected from such imagers are difficult to interpret due to a variety of “nuisance factors” in the data formation process, such as illumination, vantage point, partial occlusions, etc. These are especially severe in natural environments, where the objects of interest (e.g., plants, animals) have evolved to blend with their habitat, exhibit complex variability in shape and appearance, and perform rapid motions against dynamic backgrounds with rapid illumination changes. We describe three applications that exemplify these problems and the solutions we developed. First, we show how temporal oversampling can simplify the analysis of a slow process such as the avian nesting cycle. Then, we show how to overcome temporal undersampling in order to detect birds at a feeder station. Finally, we show how to exploit temporal consistency to reliably detect pollinators as they visit flowers in the field.

Categorization in natural time-varying image sequences

Approaches to single image categorization do not easily generalize to natural time-varying image sequences. In natural environments, object categories tend to have few features that help to distinguish between each other and the surrounding environment. To better discriminate between categories and the surrounding environment, we propose a multi-view categorization approach that exploits the statistics of image sequences rather than single images. The approach is unbiased towards redundant views - that is, it does not matter how many times an object appears from the same viewpoint. At the same time, the approach does not penalize for missing views, so that we do not have to capture an object at all viewpoints to successfully categorize the object. We first present a data set for studying natural environment monitoring: an image sequence of birds at a feeder station. After manual localization, a baseline bag of features approach was found to perform significantly worse on the proposed data set compared to the standard Caltech 101 data set. We find that our approach increases the categorization accuracy from 48% to 58% on average when compared to an equivalent single view categorization method. Finally, we show how the same metric proposed for the supervised categorization can be used to transform, in an unsupervised manner, an image sequence into a manageable set of categories.