Cem Safak Sahin

Automatic Association of Chats and Video Tracks for Activity Learning and Recognition in Aerial Video Surveillance

We describe two advanced video analysis techniques, including video-indexed by voice annotations (VIVA) and multi-media indexing and explorer (MINER). VIVA utilizes analyst call-outs (ACOs) in the form of chat messages (voice-to-text) to associate labels with video target tracks, to designate spatial-temporal activity boundaries and to augment video tracking in challenging scenarios. Challenging scenarios include low-resolution sensors, moving targets and target trajectories obscured by natural and man-made clutter. MINER includes: (1) a fusion of graphical track and text data using probabilistic methods; (2) an activity pattern learning framework to support querying an index of activities of interest (AOIs) and targets of interest (TOIs) by movement type and geolocation; and (3) a user interface to support streaming multi-intelligence data processing. We also present an activity pattern learning framework that uses the multi-source associated data as training to index a large archive of full-motion videos (FMV). VIVA and MINER examples are demonstrated for wide aerial/overhead imagery over common data sets affording an improvement in tracking from video data alone, leading to 84% detection with modest misdetection/false alarm results due to the complexity of the scenario. The novel use of ACOs and chat messages in video tracking paves the way for user interaction, correction and preparation of situation awareness reports.

Genetic algorithms for self-spreading nodes in MANETs

We present a force-based genetic algorithm for self-spreading mobile nodes uniformly over a geographical area. Wireless mobile nodes adjust their speed and direction using a genetic algorithm, where each mobile node exchanges its genetic information of speed and direction encoded in its chromosomes with the neighboring nodes. Simulation experiments show encouraging results for the performance of our force-based genetic algorithm with respect to normalized area coverage..

Multi-source Multi-modal Activity Recognition in Aerial Video Surveillance

Recognizing activities in wide aerial/overhead imagery remains a challenging problem due in part to low-resolution video and cluttered scenes with a large number of moving objects. In the context of this research, we deal with two un-synchronized data sources collected in real-world operating scenarios: full-motion videos (FMV) and analyst call-outs (ACO) in the form of chat messages (voice-to-text) made by a human watching the streamed FMV from an aerial platform. We present a multi-source multi-modal activity/event recognition system for surveillance applications, consisting of: (1) detecting and tracking multiple dynamic targets from a moving platform, (2) representing FMV target tracks and chat messages as graphs of attributes, (3) associating FMV tracks and chat messages using a probabilistic graph-based matching approach, and (4) detecting spatial-temporal activity boundaries. We also present an activity pattern learning framework which uses the multi-source associated data as training to index a large archive of FMV videos. Finally, we describe a multi-intelligence user interface for querying an index of activities of interest (AOIs) by movement type and geo-location, and for playing-back a summary of associated text (ACO) and activity video segments of targets-of-interest (TOIs) (in both pixel and geo-coordinates). Such tools help the end-user to quickly search, browse, and prepare mission reports from multi-source data.

Uniform distribution of mobile agents using genetic algorithms for military applications in MANETs

There has been increased research interest in providing uniform distribution of autonomous mobile nodes controlled by active running software agents over an unknown geographical area in mobile ad-hoc networks (MANETs). This problem becomes more challenging under the harsh and bandwidth limited conditions imposed by military applications. In this framework, the software agent running at the application layer for each autonomous mobile node adjusts its direction and speed by using local information from its neighbors. A genetic algorithm (GA) is used by each node to select the ldquofitterrdquo speed and direction options among exponentially large number of choices converging towards a uniform distribution. For a military application example, consider that in the observed occurrence of a threat situation, if the number of autonomous mobile agents change with time (e.g., losing assets during an operation), the remaining agents should reposition themselves to compensate the lost in coverage and network connectivity. We implemented simulation software to evaluate the effectiveness of GAs within these types of military applications. The results show that GAs can be applied to autonomous mobile nodes and are an effective tool for providing a robust solution for network area coverage under restrained communication conditions.

Analysis of Emergent Behavior for GA-based Topology Control Mechanism for Self-Spreading Nodes in MANETs

We introduce a genetic algorithm based MANET topology control mechanism to be used in decision making process of adaptive and autonomic systems at run time. A mobile node adapts its speed and direction using limited information collected from local neighbors operating in an unknown geographical terrain. We represent the genetic operators (i.e., selection, crossover and mutation) as a dynamical system model to describe the behavior of a single node’s decision mechanism. In this dynamical system model each mobile node is viewed as a stochastic variable. We build a homogeneous Markov chain to study the convergent nature of multiple mobile nodes running our algorithm, called FGA. Each state in our chain represents a configuration of the nodes in a MANET for a given instant. The homogeneous Markov chain model of our FGA is shown to be ergodic; its convergence is demonstrated using Dobrushin’s contraction coefficients. We also observe that the nodes with longer communication ranges utilize more information about their neighborhood to make better decisions, require less movement and converge faster, whereas smaller communication ranges utilize limited information, take more time to escape local optima, and, hence, consume more energy.

Formal convergence analysis for bio-inspired topology control in MANETs

We present a convergence analysis of a genetic algorithm based topological control mechanism for the decision making process of evolutionary and autonomous systems that adaptively reconfigures spatial configuration in mobile ad hoc networks (MANETs). Mobile nodes adjust their speed and direction using information collected from the local neighborhood environment in unknown geography. We extend the stochastic model of the genetic operators (i.e., selection, crossover and mutation) called the dynamical system model that represents the behavior of a single nodes decision mechanism in the network viewed as a stochastic variable. We introduce an ergodic homogeneous Markov chain to analyze the convergent nature of multiple mobile nodes running our algorithm, called the Force-based Genetic Algorithm (FGA). Here, a state represents an instantaneous spatial configuration of nodes in a MANET. It is shown that the Markov chain model of our FGA is ergodic and its convergence is shown using Dobrushins contraction coefficients. It is observed that scenarios where nodes have small communication ranges compared to their movement range converge quicker than larger ones due the limited information they have of their neighborhood, making movement decisions simpler, thus conserving energy.

Estimating behavior of a GA-based topology control for self-spreading nodes in MANETs

This paper presents a dynamical system model for FGA, a force-based genetic algorithm, which is used as decentralized topology control mechanism among active running software agents to achieve a uniform spread of autonomous mobile nodes over an unknown geographical area. Using only local information, FGA guides each node to select a fitter location, speed and direction among exponentially large number of choices, converging towards a uniform node distribution. By treating a genetic algorithm (GA) as a dynamical system we can analyze it in terms of its trajectory in the space of possible populations. We use Voses theoretical model to calculate the cumulative effects of GA operators of selection, mutation, and crossover as a population evolves through generations. We show that FGA converges toward a significantly higher area coverage as it evolves.

Convergence analysis of genetic algorithms for topology control in MANETs

We describe and verify convergence properties of our forced-based genetic algorithm (FGA) as a decentralized topology control mechanism distributed among software agents. FGA uses local information to guide autonomous mobile nodes over an unknown geographical terrain to obtain a uniform node distribution. Analyzing the convergence characteristics of FGA is difficult due to the stochastic nature of GA-based algorithms. Ergodic homogeneous Markov chains are used to describe the convergence characteristics of our FGA. In addition, simulation experiments verify the convergence of our GA-based algorithm.

Markov chain model for differential evolution based topology control in MANETs

Mobile Ad hoc Networks (MANETs) are used for many strategic commercial and military applications where it is not feasible to use a centralized controller or manually deploy assets. They have proved useful for many practical applications, such as search and rescue, clearing mine fields, and transportation systems. We introduce a differential evolution based topological control mechanism for the decision making process of evolutionary and autonomous systems that adaptively reconfigures spatial configuration in MANETs. We present a formal analysis of the effectiveness of our topology control mechanism and introduce an inhomogeneous Markov chain model to prove its convergence. The experiment results from our simulation software show that our biologically-inspired algorithm produces encouraging results for uniform distribution of mobile nodes over unknown terrains.

Resilient node self-positioning methods for MANETS based on game theory and genetic algorithms

We present a distributed and scalable game participated by autonomous MANET nodes to place themselves uniformly over a dynamically changing environment. A node spreading potential game, called Rel-NSPG, run at each node, autonomously makes movement decisions based on localized data while the best next location to move is selected by a genetic algorithm (GA). Since it requires only a limited synchronization among the closest neighbors of a player, and does not require a priori knowledge of the environment, Rel-NSPG is a good candidate for node spreading class of applications used in military tasks. The performance of Rel-NSPG degrades gracefully when the number of MANET nodes decrease either due to equipment malfunction or hostile activities. We show that this resilience to loss of nodes is inherent in Rel-NSPG. Simulation experiments demonstrate that, after a subset of the MANET nodes arbitrarily become unavailable, the remaining nodes recover and offset lost nodes. Similarly, when there are losses concentrated in a given region, remaining nodes reconfigure their positions to compensate for the missing area coverage. The simulation experiments with arbitrarily placed obstacles, in addition to lost assests, produce promising results.