Carlos Vallespi

CAMEO: Camera Assisted Meeting Event Observer

Static cameras are pervasive in a variety of environments. However it remains a challenging problem to extract and reason about high-level features from real-time and continuous observation of an environment. In this paper, we present CAMEO, the Camera Assisted Meeting Event Observer, which is a physical awareness system designed for use by an agent-based electronic assistant. CAMEO is an inexpensive high-resolution omnidirectional vision system designed to be used in meeting environments. The multiple camera design achieves the desired high image resolution and lower cost that can be achieved when compared to traditional omnicameras that make use of a single camera and mirror solution.

Automatic Clustering of Faces in Meetings

Meetings are an integral part of business life for any organization. In previous work, we have developed a physical awareness system called CAMEO (camera assisted meeting event observer) to record and process the audio/visual information of a meeting. An important task in meeting understanding is to know who and how many people are attending the meeting. In this paper, we present an automatic approach to detect, track, and cluster peoples faces in long video sequences. This is a challenging problem due to the appearance variability of peoples faces (illumination, expression, pose,...). Two main novelties are presented: a robust real-time adaptive subspace face tracker which combines color and appearance. A temporal subspace clustering algorithm. The effectiveness and robustness of the proposed system is demonstrated over a data set of long videos (i.e. 1 hour).

PVS: A system for large scale outdoor perception performance evaluation

This paper describes the motivation, design and implementation of a Perception Validation System (PVS), a system for measuring the outdoor perception performance of an autonomous vehicle. The PVS relies on using large amounts of real world data and ground truth information to quantify performance aspects such as the rate of false positive or false negative detections of an obstacle detection system. Our system relies on a relational database infrastructure to achieve a high degree of flexibility in the type of analyses it can support.

Learning to Track Multiple People in Omnidirectional Video

Meetings are a very important part of everyday life for professionals working in universities, companies or governmental institutions. We have designed a physical awareness system called CAMEO (Camera Assisted Meeting Event Observer), a hardware/software system to record and monitor peoples activities in meetings. CAMEO captures a high resolution omnidirectional view of the meeting by stitching images coming from almost concentric cameras. Besides recording capability, CAMEO automatically detects people and learns a person-specific facial appearance model (PS-FAM) for each of the participants. The PSFAMs allow more robust/reliable tracking and identification. In this paper, we describe the video-capturing device, photometric/geometric autocalibration process, and the multiple people tracking system. The effectiveness and robustness of the proposed system is demonstrated over several real-time experiments and a large data set of videos.

Segmentation and classification of meetings using multiple information streams

We present a meeting recorder infrastructure used to record and annotate events that occur in meetings. Multiple data streams are recorded and analyzed in order to infer a higher-level state of the groups activities. We describe the hardware and software systems used to capture peoples activities as well as the methods used to characterize them.

People in the weeds: Pedestrian detection goes off-road

Robotics offers a great opportunity to improve efficiency while also improving safety, but reliable detection of humans in off-road environments remains a key challenge. We present a person detector evaluation on a dataset collected from an autonomous tractor in an off-road environment representing challenging conditions with significant occlusion from weeds and branches as well as non-standing poses. We apply three image-only algorithms from urban pedestrian detection to better understand how well these approaches work in this domain. We evaluate the Aggregate Channel Features (ACF) and Deformable Parts Model (DPM) algorithms from the literature, as well as our own implementation of a Convolutional Neural Network (CNN). We show that the traditional performance metric used in the pedestrian detection literature is extremely sensitive to parameterization. When applied in domains like this one, where localization is challenging due to high background texture and occlusion, the choice of overlap threshold strongly affects measured performance. Using a permissive overlap threshold, we found that ACF, DPM, and CNN perform similarly overall in this domain, although they each have different failure modes.