Nikos Papanikolopoulos

Vision-based methods for driver monitoring

This project involves the use of a dashboard-mounted camera to monitor the direction a driver is looking. This is accomplished by using the framework for processing video (FPV), developed at the University of Minnesota by Osama Masoud. The monitoring software works by first finding the lips on the drivers face using color analysis. Then, the skin color of the driver can be sampled and a face region is generated. The largest holes in this region are the lips and eyes. Once the eyes are found, the software finds the darkest pixels in the eyes and marks these as the pupils. The software uses the relative positions of the eyes and pupils to make statements about the gaze direction. Also discussed is monitoring various other driver activities such as adjusting as car stereo.

Practical mixtures of Gaussians with brightness monitoring

We discuss some of the practical issues concerning the use of mixtures of Gaussians for background segmentation in outdoor scenes, including the choice of parameters. Different covariance representations and their performance impact are examined. In addition, we propose a simple, yet efficient method for coping with sudden global illumination changes based on smoothing brightness and contrast changes over time. All of the discussed methods are capable of running in real time at reasonable resolution on current generation PCs.

Enabling complex behavior by simulating marsupial actions

Marsupial robotics is a field which requires a high initial investment prior to experiments. We present a new simulation tool used for marsupial actions, enabling research in the area of marsupial robotics. A flexible abstraction for a marsupial system is presented which allows for the modeling of many differing marsupial models. The interface and simulation model are covered in detail. Two experiments are presented which use the model to demonstrate the flexibility and range of the abstraction and simulation.

Motion estimation of a miniature helicopter using a single onboard camera

This paper describes a technique for the estimation of the translational and rotational velocities of a miniature helicopter using the video signals from a single onboard camera. For every two consecutive frames from the camera, point correspondences are identified and Epipolar Geometry based algorithms are used to find the likely estimates of the absolute rotations and relative displacements. Images from onboard camera are often corrupted with various types of noises; SIFT descriptors were found to be the best feature descriptors to be used for point correspondences. To speed up the processing, we introduce a new representation of these descriptors based on compressive sensing formalisms. To estimate the absolute displacement of the helicopter between frames, we use the measurements from a simulated IR sensor to find the true change in altitude of the body frame, scaling other translational dimensions accordingly, and later estimating the velocities. Experiments conducted using data from a real helicopter in an indoor environment demonstrate promising results.

Miniature embedded rotorcraft platform for aerial teleoperation experiments

This paper describes the hardware and software systems that make up an experimental architecture developed to facilitate teleoperation research for miniature rotorcraft. One objective in developing this platform is to place the minimum required sensors and electronics onboard a miniature helicopter with limited payload. The onboard hardware resources must be sufficient to simplify the teleoperation task for the operator. While being developed, the onboard system is coupled with the infrastructure at the Interactive Guidance and Control Lab at the University of Minnesota to allow for a systematic approach to the investigation of onboard control and guidance augmentations. The overall architecture integrates components that allow a variety of estimation, control and guidance algorithms to be conveniently implemented and tested. In the lab environment, a Vicon tracking system is used to accurately determine a helicopters pose and orientation and evaluate algorithms on stationary computers. This accurate information is used to develop the necessary metrics and evaluate the performance of different algorithms before transition to real-world situations.

Last-Mile Transit Service with Urban Infrastructure Data

In this article, we propose a transit service Feeder to tackle the last-mile problem, that is, passengers’ destinations lay beyond a walking distance from a public transit station. Feeder utilizes ridesharing-based vehicles (e.g., minibus) to deliver passengers from existing transit stations to selected stops closer to their destinations. We infer real-time passenger demand (e.g., exiting stations and times) for Feeder design by utilizing extreme-scale urban infrastructures, which consist of 10 million cellphones, 27 thousand vehicles, and 17 thousand smartcard readers for 16 million smartcards in a Chinese city, Shenzhen. Regarding these numerous devices as pervasive sensors, we mine both online and offline data for a two-end Feeder service: a back-end Feeder server to calculate service schedules and front-end customized Feeder devices in vehicles for real-time schedule downloading. We implement Feeder using a fleet of vehicles with customized hardware in a subway station of Shenzhen by collecting data for 30 days. The evaluation results show that compared to the ground truth, Feeder reduces last-mile distances by 68% and travel time by 56%, on average.

Socializing Surveillance: An interdisciplinary educational model

Surveillance systems are becoming ubiquitous nowadays. Their use is sometimes followed by misuse which often raises ethical questions that have implications for all involved in the design, control, and implementation of surveillance schemes. Spanning across the disciplines of interior design/architecture and camera networks present conceptions of surveillance systems often disregard the social dimension, or those ‘being watched.’ Debates raise concerns as people associate cameras with feelings of fear and control, loss of privacy, discrimination, inequality, and cultural/gender insensitivity. Began as an interdisciplinary collaboration between the University of Minnesotas Interior Design program and the Department of Computer Science and Engineering, the ‘Socializing Surveillance’ project matured into a joint endeavor undertaken by four faculty advisors and five electrical and computer engineering students through the “ECE 4951: Senior Design” Project in Fall 2009 and Spring 2010. This paper outlines the pertinent issues starting from an optimization approach to place cameras and in-depth interviews with design practitioners that led to a re-thinking of what surveillance and camera networks can entail; the challenges and opportunities afforded through interdisciplinary educational efforts; and the attempts of the team to develop a prototype for an interactive surveillance system that foregrounds the social dimensions of a security scheme. The goal is to ignite interest in surveillance and to set in place considerations for interdisciplinary educational models.

Learning Discriminative αβ-Divergences for Positive Definite Matrices

Symmetric positive definite (SPD) matrices are useful for capturing second-order statistics of visual data. To compare two SPD matrices, several measures are available, such as the affine-invariant Riemannian metric, Jeffreys divergence, Jensen-Bregman logdet divergence, etc.; however, their behaviors may be application dependent, raising the need of manual selection to achieve the best possible performance. Further and as a result of their overwhelming complexity for large-scale problems, computing pairwise similarities by clever embedding of SPD matrices is often preferred to direct use of the aforementioned measures. In this paper, we propose a discriminative metric learning framework, Information Divergence and Dictionary Learning (IDDL), that not only learns application specific measures on SPD matrices automatically, but also embeds them as vectors using a learned dictionary. To learn the similarity measures (which could potentially be distinct for every dictionary atom), we use the recently introduced αß-logdet divergence, which is known to unify the measures listed above. We propose a novel IDDL objective, that learns the parameters of the divergence and the dictionary atoms jointly in a discriminative setup and is solved efficiently using Riemannian optimization. We showcase extensive experiments on eight computer vision datasets, demonstrating state-of-the-art performances.