Daniel Henkel

Towards autonomous data ferry route design through reinforcement learning

Communication in delay tolerant networks can be facilitated by the use of dedicated mobile ldquoferriesrdquo which physically transport data packets between network nodes. The goal is for the ferry to autonomously find routes which minimize the average packet delay in the network. We prove that paths which visit all nodes in a round-trip fashion, i.e., solutions to the traveling salesman problem, do not yield the lowest average packet delay. We propose two novel ferry path planning algorithms based on stochastic modeling and machine learning. We model the path planning task as a Markov decision process with the ferry acting as an independent agent. We apply reinforcement learning to enable the ferry to make optimal decisions. Simulation experiments show the resulting routes have lower average packet delay than solutions known to date.

Sensor Data Collection Through Gateways in a Highly Mobile Mesh Network

Widely distributed sensors must discover paths back to data collection points possibly through sparsely connected and mobile networks. Current addressing and service discovery schemes in mobile networks are not well-suited to multihop disconnected networks. This paper describes an architecture and protocol for sensor data collection through highly mobile ad-hoc network (MANET) that may never experience end-to-end connectivity. Special gateway nodes are described which are responsible for intelligently routing messages to their intended destination(s). These gateway nodes qualify their links and announce their status to the MANET, a simple approach to service discovery that is effective in this implementation. The protocol is implemented and tested in a laboratory and outdoor environment.

A reliable sensor data collection network using unmanned aircraft

This paper presents a method for reliably collecting data events from sensors and forwarding the data via a MANET to sensor monitoring stations located on an external network. A the core is a MANET concept that consists of ground and unmanned aircraf nodes. Unmanned aircraf enable a model whereby widely-spaced sensors are intermittently connected to the network and data is sent in stages as connections become available along each stage. The paper describes the sensor data collection model, the reliable multicast data delivery mechanism, and our experiences on a network test bed including the control of an unmanned aircraft through a MANET.

Phase Transitions for Controlled Mobility in Wireless Ad hoc Networks

*† ‡ § In this paper we investigate the phase transitions between different modes of controlled mobility in wireless ad hoc networks. The transmission of data between two nodes can be performed by one of three methods in a mobile ad hoc network: direct transmission between nodes; multi-hop relaying through intermediate nodes; and data ferrying through a node that physically moves between sources and destinations. Assuming the source and destination nodes are stationary, the best choice of transmission mode through the network is a function of several variables including the separation distance between nodes, the required average data throughput, the maximum tolerable delay, and the data ferry speed and buffer capacity. This paper presents the notion of a phase diagram relating separation distance to average data throughput. Contours of maximum packet delay are specified on this phase plot and are used to identify the optimal mode under various configurations. The creation and maintenance of communication channels of specified performance in a mobile ad hoc network can be viewed as a hybrid control problem. Calculation of the boundaries between phases corresponds to determining the transition conditions of the hybrid controller.

Establishment and Maintenance of a Delay Tolerant Network through Decentralized Mobility Control

In this paper we investigate the establishment and maintenance of a delay tolerant network through decentralized mobility control. A set of geographically dispersed wireless radio nodes wish to communicate with one another. These nodes are free to move about the environment and are not constrained to remain in direct communication range of one another. A second set of helper nodes exist in the environment that can form connected chains between nodes and that can ferry data back and forth between sensor nodes and/or other helper nodes. The mobility of the helper nodes is controlled based on local information about neighboring nodes and the communication flow through the network. A hierarchical approach is described in which helper nodes are assigned specific sensor nodes or regions of activity through resource allocation algorithms while communication flow through established links is maintained by decentralized control based on local measures only such as received signal strength or signal to noise ratio

Delay-tolerant communication using mobile robotic helper nodes

Delay tolerant networks can exhibit long phases of complete unconnectedness, where several nodes will not be able to communicate effectively. In those circumstances controlling the movement of helper nodes, that facilitate message exchanges instead of relying on direct communication, can improve network performance. We investigate when controlled movement is beneficial and how helper nodes need to move or position themselves to achieve maximum network performance. Three communication modes on a single link and in a hub model are analyzed: direct communication, communication through one or more relays, and communication via data ferries. Hereby we extend the traditional in-range, out-of-range communication model to include a distance dependence which gives a differentiated rate-distance profile. Further, we characterize controlled mobility of mobile data ferries to form static relay paths or dynamically ferry data between nodes. Achievable throughput, delay, and distance performance of the link modes change with channel and ferry characteristics. The union of these performance regions forms the achievable communication space for a given DTN scenario. Novel phase plots show when it is feasible to use one of the mobility schemes versus direct communication. The implications of these modes for practical link-layer designs are discussed using a real-world DTN example of sensor data collection with unmanned aircraft.

Optimizing the Use of Relays for Link Establishment in Wireless Networks

Multihop packet relaying has been shown to extend the achievable communication space in sparsely connected ad hoc networks, in sensor networks, and over long distance links. This paper investigates the properties of relay-enabled networks as a function of the number of relays and whether packets are sent one at a time or whether multiple packets are simultaneously sent along the relay chain. When packets are sent one at a time, an optimal number of relays is found that maximizes the throughput and minimizes delay. The optimal number is in terms of a standard hop, which enables the number of relays,maximum throughput, and minimum delay to be determined for any communication within the environment. When multiple packets are sent at a time, a maximum throughput is found that is independent of distance and noise. The performance depends on the separation in hops between simultaneous transmissions in the relay chain. The optimal separation is found in the limit of many relays to be 5 hops across many different radio environments. These results allow us to define feasible delay-throughput-distance performance regions and provide inputs to better multi-hop wireless network design.

Sensor Data Collection through Unmanned Aircraft Gateways

Current addressing and service discovery schemes in mobile networks are not well-suited to multihop disconnected networks. This paper describes an implementation of a highly mobile ad-hoc network (MANET) that may never experience end-to-end connectivity. Special gateway nodes are described which are responsible for intelligently routing messages to their intended destination(s). These gateway nodes qualify their links and announce their status to the MANET, a simple approach to service discovery that is effective in this implementation. This implementation has been tested in an outdoor environment.

Route Design for UAV-based Data Ferries in Delay Tolerant Wireless Networks

The Ferrying communication model has been proposed as a viable means of enabling delay-tolerant packet transport in intermittently connected networks. In this paper we analyze algorithms for finding an optimal ferry route through a network of stationary task nodes with the goal of minimizing the average packet delay. We show the complexity of the planning problem and prove that a simple Traveling Salesman-based approach does not always yield the optimal solution. We propose to frame the problem in two alternative ways. Our first algorithm is based on queueing theory and gives a probabilistic solution for a static environment. In the second approach we model the ferry route planning problem as a Markov Decision Process (MDP) with finite state and action spaces to capture system variability. Reinforcement learning (RL) can then be applied to find a solution. Preliminary results show that in selected example networks these two algorithms might be able to obtain routes which are superior to routes found using commonly known methods.