Markus Quaritsch

Autonomous multicamera tracking on embedded smart cameras

There is currently a strong trend towards the deployment of advanced computer vision methods on embedded systems. This deployment is very challenging since embedded platforms often provide limited resources such as computing performance, memory, and power. In this paper we present a multicamera tracking method on distributed, embedded smart cameras. Smart cameras combine video sensing, processing, and communication on a single embedded device which is equipped with a multiprocessor computation and communication infrastructure. Our multicamera tracking approach focuses on a fully decentralized handover procedure between adjacent cameras. The basic idea is to initiate a single tracking instance in the multicamera system for each object of interest. The tracker follows the supervised object over the camera network, migrating to the camera which observes the object. Thus, no central coordination is required resulting in an autonomous and scalable tracking approach. We have fully implemented this novel multicamera tracking approach on our embedded smart cameras. Tracking is achieved by the well-known CamShift algorithm; the handover procedure is realized using a mobile agent system available on the smart camera network. Our approach has been successfully evaluated on tracking persons at our campus.

The evolution from single to pervasive smart cameras

Having seen increased interest from the research community, smart camera systems have gone through a number of evolutionary steps like from single cameras to distributed smart camera systems with collaboration features. This work aims at defining a taxonomy to classify these systems based on their platform capabilities, the degree of distributed processing as well as system autonomy aspects covering self-configuration and mobility. Existing camera systems are classified according to the proposed taxonomy. Besides capturing the design space for smart cameras, the main contribution of this paper is an overview of the research challenges for the envisioned class of pervasive smart camera systems. As defined in this work, future pervasive smart camera systems will be visual sensor networks targeted at end-user applications where special emphasis is put on unobtrusiveness of the cameras as well as simple deployment supported by self configuration capabilities.

Embedded Middleware on Distributed Smart Cameras

Two trends emerge in recent image processing research: distributed computing and embedded processing. Both trends are exemplified in smart cameras which combine image sensing, image processing and communication on a single embedded device. Networks of distributed smart cameras help to overcome some hard problems that are inherent in single-camera systems. Designing, implementing and deploying image processing applications for cooperating distributed cameras is much more complex than for single-camera systems. In this paper, we focus on software services required for distributed embedded image processing on a network of smart cameras. We identify important middleware services and present our lightweight middleware implemented on our distributed SmartCams. A multi-camera tracking application demonstrates the benefits of our approach.

Integrating multi-camera tracking into a dynamic task allocation system for smart cameras

This paper reports on the integration of multi-camera tracking into an agent-based framework, which features autonomous task allocation for smart cameras targeting traffic surveillance. Since our target platforms are distributed embedded systems with limited resources, the trackers may only be active, if the target is in the cameras field of view. Consequently, the tracking algorithm has to migrate from camera to camera in order to follow the target, whereas the decision when and whereto the migration takes place is reached autonomously by the tracker. Consequently, no central control host is required. We further present different strategies on when to migrate a tracker, and how to determine the camera which observes the tracked object subsequently. We have realized the trackers control by using heterogeneous mobile agents, which employ a state-of-the-art tracking algorithm for tracking. The tracking system has been implemented on our smart cameras (SmartCam) which are comprised of a network processor and several digital signal processors (DSPs) and provide a complex software framework.

On path planning strategies for networked unmanned aerial vehicles

In this paper, we compare deterministic and probabilistic path planning strategies for an autonomous unmanned aerial vehicle (UAV) network, where the objective is to explore a given area with obstacles and provide an overview image. We present both online and offline implementations of the algorithms as alternative solutions, where applicable, and analyze the performance of the offline implementations. Results illustrate the benefits and drawbacks of different planning strategies and provide insight into which strategy should be taken, given the constraints of the application of interest.

Improved Agent-Oriented Middleware for Distributed Smart Cameras

In the recent past, much effort has been put into the development of distributed vision systems with smart cameras as key components. Smart cameras combine video sensing, processing and communication within a single embedded device and provide sufficient on-board infrastructure to carry out high-level video analysis tasks. Networks of smart cameras help to overcome some hard problems inherent to single-camera systems by providing multiple views of a scene. This paper reports on an improved, agent-oriented middleware for embedded smart cameras. Each image processing task is represented by an agent resident on a smart camera within the network. Agents are able to move from one camera to another as needed during run-time. An agent is comprised of the high-level application logic and the image processing algorithm which is executed on the processing unit. The presented middleware is also designed for distributed image processing where two or more cameras can cooperate for a single task. In the paper we discuss the requirements for such an agent-oriented middleware capable of supporting distributed image processing. Further, we describe the architecture of our middleware implementation. The evaluation of our current middleware implementation shows significant performance improvements compared to our previous Java-based implementation.

Incremental, orthorectified and loop-independent mosaicking of aerial images taken by micro UAVs

In this paper we survey thoroughly the problem of orthorectified and incremental image mosaicking of a sequence of aerial images taken from low-altitude micro aerial vehicles. Most of existing approaches have been exploiting the global optimization (in presence of a loop in the image sequences) to distribute and/or metadata to mitigate the accumulating stitching error. However, the resulting mosaic can be improved if the errors are diminished by studying their sources. Mostly the UAV aerial image mosaicking is affected by the following three important sources of error: i) a weak homography as a result of using unleveled ground control points (GCPs) for image registration, ii) a poor camera calibration and image rectification, and iii) deficiency of a well-defined projection model (cylindrical, planar, etc) and consequently an inappropriate transformation model. We investigate the influences of using a depth map to find the features from the same plane, geometric distortion correction and combining the appropriate choice of projection and transformation model for the mosaicking. We further quantify the improvement of orthorectification in mosaics by mitigating those errors and demonstrate the improvement on real-world mosaics.

Embedded Middleware for Smart Camera Networks and Sensor Fusion

Smart cameras represent an interesting research field that has evolved over the last decade. In this chapter we focus on the integration of multiple, potentially heterogeneous smart cameras into a distributed system for computer vision and sensor fusion. Because an important aspect of distributed systems is the system-level software, also called middleware, we discuss the middleware requirements of distributed smart cameras and the services the middleware must provide. In our opinion, a middleware following the agent-oriented paradigm allows us to build flexible and self-organizing applications that encourage a modular design.

Communication and Coordination for Drone Networks

Small drones are being utilized in monitoring, delivery of goods, public safety, and disaster management among other civil applications. Due to their sizes, capabilities, payload limitations, and limited flight time, it is not far-fetched to expect multiple networked and coordinated drones incorporated into the air traffic. In this paper, we describe a high-level architecture for the design of a collaborative aerial system that consists of drones with on-board sensors and embedded processing, sensing, coordination, and communication&networking capabilities. We present a multi-drone system consisting of quadrotors and demonstrate its potential in a disaster assistance scenario. Furthermore, we illustrate the challenges in the design of drone networks and present potential solutions based on the lessons we have learned so far.

DSCAgents: A lightweight middleware for distributed smart cameras

This paper investigates on middleware for distributed smart cameras. We describe DSCAgents, our agent-oriented lightweight middleware system. The design goal is to provide a modular and flexible middleware that takes into account the underlying hardware platform and also supports collaborative image processing. Mobile agents are used to model image processing tasks and to manage the whole smart camera network. The agent-oriented approach simplifies application development and domain specific services unburden programmers from implementing the same functionality for each application again. Mobility of agents allows to build highly dynamic and adaptive systems where image processing tasks can move from one camera to another during operation. The evaluation of DSCAgents shows reasonable performance while keeping the resource requirements low.