Harini Veeraraghavan

Computer vision algorithms for intersection monitoring

The goal of this project is to monitor activities at traffic intersections for detecting/predicting situations that may lead to accidents. Some of the key elements for robust intersection monitoring are camera calibration, motion tracking, incident detection, etc. In this paper, we consider the motion-tracking problem. A multilevel tracking approach using Kalman filter is presented for tracking vehicles and pedestrians at intersections. The approach combines low-level image-based blob tracking with high-level Kalman filtering for position and shape estimation. An intermediate occlusion-reasoning module serves the purpose of detecting occlusions and filtering relevant measurements. Motion segmentation is performed by using a mixture of Gaussian models which helps us achieve fairly reliable tracking in a variety of complex outdoor scenes. A visualization module is also presented. This module is very useful for visualizing the results of the tracker and serves as a platform for the incident detection module.

Robust target detection and tracking through integration of motion, color, and geometry

Vision-based tracking is a basic elementary task in many computer vision-based applications such as video surveillance and monitoring, sensing and navigation in robotics, video compression, video annotation, and many more. However, reliable recovery of targets and their trajectories in an uncontrolled environment is affected by a wide range of conditions exhibited by the environment such as sudden illumination changes and clutter. This work addresses the problem of (i) combining information from a set of cues in order to obtain reasonably accurate estimates of multiple targets in uncontrolled environments and (ii) a collection of data association methods for cues containing less information for robust tracking through persistent clutter. Specifically, we introduce a novel geometric template constrained data association method for robust tracking of point features, while using the Joint Probabilistic Data Association (JPDA) method for blob cue measurements. Extensive experimental validation of the tracking and the data association framework is presented in the work for several real-world outdoor traffic intersection image sequences.

Communication Strategies in Multi-robot Search and Retrieval: Experiences with MinDART

To explore the effects of different simple communications strategies on performance of robot teams, we have conducted a set of foraging experiments using real robots (the Minnesota Distributed Autonomous Robotic Team). Our experimental results show that more complex communication strategies do not necessarily improve task completion times, but tend to reduce variance in performance.

Driver activity monitoring through supervised and unsupervised learning

This paper presents two different learning methods applied to the task of driver activity monitoring. The goal of the methods is to detect periods of driver activity that are not safe, such as talking on a cellular telephone, eating, or adjusting the dashboard radio system. The system presented here uses a side-mounted camera looking at a drivers profile and utilizes the silhouette appearance obtained from skin-color segmentation for detecting the activities. The unsupervised method uses agglomerative clustering to succinctly represent driver activities throughout a sequence, while the supervised learning method uses a Bayesian eigen-image classifier to distinguish between activities. The results of the two learning methods applied to driving sequences on three different subjects are presented and extensively discussed.

Learning Dynamic Event Descriptions in Image Sequences

Automatic detection of dynamic events in video sequences has a variety of applications including visual surveillance and monitoring, video highlight extraction, intelligent transportation systems, video summarization, and many more. Learning an accurate description of the various events in real-world scenes is challenging owing to the limited user-labeled data as well as the large variations in the pattern of the events. Pattern differences arise either due to the nature of the events themselves such as the spatio-temporal events or due to missing or ambiguous data interpretation using computer vision methods. In this work, we introduce a novel method for representing and classifying events in video sequences using reversible context-free grammars. The grammars are learned using a semi-supervised learning method. More concretely, by using the classification entropy as a heuristic cost function, the grammars are iteratively learned using a search method. Experimental results demonstrating the efficacy of the learning algorithm and the event detection method applied to traffic video sequences are presented.

Performance Evaluation of a Multi-Robot Search & Retrieval System: Experiences with MinDART

Swarm techniques, where many simple robots are used instead of complex ones for performing a task, promise to reduce the cost of developing robot teams for many application domains. The challenge lies in selecting an appropriate control strategy for the individual units. This work explores the effect of control strategies of varying complexity and environmental factors on the performance of a team of robots at a foraging task when using physical robots (the Minnesota Distributed Autonomous Robotic Team). Specifically we study the effect of localization and of simple indirect communication techniques on task completion time using two sets of foraging experiments. We also present results for task performance with varying team sizes and target distributions. As indicated by the results, control strategies with increasing complexity reduce the variance in the performance, but do not always reduce the time to complete the task.

Switching Kalman Filter-Based Approach for Tracking and Event Detection at Traffic Intersections

Automatic event detection from video sequences has applications in several areas such as automatic visual surveillance, traffic monitoring for intelligent transportation systems, key frame detection for video compression, and virtual reality applications. In this work, we present a computer vision-based approach for event detection and data collection at traffic intersections. Specifically, we make the following two contributions: (i) a robust tracking algorithm for targets through combination of multiple cues and multiple motion models, and (ii) a simple event detection system using the results of a switching Kalman filter in combination with some simple rules. We show the results of tracking and event detection, such as, turning, stopped or stalling vehicles, as well as motion statistics (average speeds and accelerations), collected for some outdoor traffic scenes

Learning to Recognize Video-Based Spatiotemporal Events

A key research issue in activity recognition in real-world applications, such as in intelligent transportation systems (ITS), is to automatically learn robust models of activities that require minimal human training. In this paper, we contribute a novel approach for learning sequenced spatiotemporal activities in outdoor traffic intersections. Concretely, by representing the activities as sequences of actions, we contribute a semisupervised learning algorithm that learns activities as complete stochastic context-free grammars (SCFGs), namely, the grammar structure and the parameters. Our approach has been implemented and tested on real-world scenes, and we present experimental results of the grammar learning and activity recognition applied to data collection and traffic monitoring applications using video data.

No fear: University of Minnesota Robotics Day Camp introduces local youth to hands-on technologies

Women and minorities are underrepresented in the IT field at the high school, university, and industry levels. Efforts to address this imbalance are often too late to solve underlying problems such as perceived ineptitude and actual inexperience. By designing and hosting a program for these underrepresented students in the middle grades, the Center for Distributed Robotics at the University of Minnesota hopes to establish a successful annual robotics day camp which would inspire both women and minorities to pursue careers in technology. Detailed accounts of the goals and methodology are provided. Initial survey results reveal a very positive response from the campers as well as strengths and weaknesses which would be useful in designing or refining similar camps

Using Robots to Raise Interest in Technology Among Underrepresented Groups

Women and minorities are under represented in the IT field at the high school, university, and industry levels. Efforts to address this imbalance are often too late to solve underlying problems such as perceived ineptitude and actual inexperience. By designing and hosting a program for these underrepresented students in the middle grades, the Center for Distributed Robotics at the University of Minnesota hopes to establish a successful annual robotics day camp that will inspire both women and minorities to pursue careers in technology. Detailed accounts of the goals and methodology are provided. Initial survey results reveal a very positive response from the campers as well as strengths and weaknesses that will be useful in designing or refining similar camps.