Carl Fossa

Competing Mobile Network Game: Embracing antijamming and jamming strategies with reinforcement learning

We introduce Competing Mobile Network Game (CMNG), a stochastic game played by cognitive radio networks that compete for dominating an open spectrum access. Differentiated from existing approaches, we incorporate both communicator and jamming nodes to form a network for friendly coalition, integrate antijamming and jamming subgames into a stochastic framework, and apply Q-learning techniques to solve for an optimal channel access strategy. We empirically evaluate our Q-learning based strategies and find that Minimax-Q learning is more suitable for an aggressive environment than Nash-Q while Friend-or-foe Q-learning can provide the best solution under distributed mobile ad hoc networking scenarios in which the centralized control can hardly be available.

Internetworking tactical MANETs

Mobile Ad-Hoc Networks (MANETS) will play a significant role in future tactical military networks. Tactical networks are required to support military operations in areas without access to a fixed network infrastructure. They are also characterized by frequent changes in network topology due to node mobility and intermittent line-of-sight (LOS) connectivity. The self-forming and self-healing nature of MANETs is therefore advantageous in a tactical military network. This is evidenced by programs such as the Joint Tactical Radio System (JTRS) and Warrior Information Network – Tactical (WIN-T), which are developing MANET capable radios and waveforms for future military operations. Most work to date addresses the challenges of networking and scalability within a MANET. The internetworking of MANETs presents a set of challenges which to date have been largely overlooked. The general assumption is that techniques and protocols currently used to interconnect fixed networks in the Internet will work equally well for MANETs. However, protocols like Border Gateway Protocol (BGP), which is widely used to interconnect autonomous networks in the Internet, may not be well suited to address the dynamics of networking between MANETs. There are a number of approaches to managing the challenges associated with internetworking MANETs, e.g. abstracting route information between MANETs, managing and updating connection points, addressing network partitions, etc. Each of these approaches offers a different balance between limiting the exchange of routing information generated by node mobility, and maintaining up-to-date routes to all nodes. The solution space can be divided into routing techniques and mobility management techniques. This paper presents simulation studies which examine the trade-offs between scalability and reachability when interconnecting tactical MANETs.

On heterogeneous mobile network connectivity: Number of gateway nodes

A mobile tactical network is characterized by wireless communication nodes operating over a disperse geographical area. As tactical nodes move during an operation, the network may partition into several segregated clusters. Once the network has partitioned, mobile nodes in different clusters cannot maintain connectivity due to insufficient radio transmission range. The partitioned network will have limited capability in providing seamless communication services to sensors and combat systems. To mitigate this problem, a subset of the mobile nodes can be collocated with and connected to a more powerful communication node to form a gateway node. These more powerful nodes have longer radio transmission range and are assumed to be connected with each other to form an upper tier network (e.g., satellite network). To reach its destination mobile node through nodes in the upper tier network, a regular mobile node can first connect to a gateway node. The gateway node can then forward traffic through the connected upper tier network to another gateway node. In this scenario, communication between mobile nodes in different clusters can only occur when each cluster contains a gateway node. In this paper, we investigated the number of gateway nodes needed in order to maintain certain level of connectivity in a mobile network. Given the node density of the mobile network, we quantified the relationship between network connectivity and the number of gateway nodes. In a densely populated mobile network, we found that only a small number of gateway nodes are needed to achieve good network connectivity. Moreover, as the node density increases, the percentage of gateway nodes can decrease at a larger rate than the node density increase rate while still achieving a good network connectivity.

Competing Cognitive Resilient Networks

We introduce competing cognitive resilient network (CCRN) of mobile radios challenged to optimize data throughput and networking efficiency under dynamic spectrum access and adversarial threats (e.g., jamming). Unlike the conventional approaches, CCRN features both communicator and jamming nodes in a friendly coalition to take joint actions against hostile networking entities. In particular, this paper showcases hypothetical blue force and red force CCRNs and their competition for open spectrum resources. We present state-agnostic and stateful solution approaches based on the decision theoretic framework. The state-agnostic approach builds on multiarmed bandit to develop an optimal strategy that enables the exploratory-exploitative actions from sequential sampling of channel rewards. The stateful approach makes an explicit model of states and actions from an underlying Markov decision process and uses multiagent Q-learning to compute optimal node actions. We provide a theoretical framework for CCRN and propose new algorithms for both approaches. Simulation results indicate that the proposed algorithms outperform some of the most important algorithms known to date.

Fast Online Learning of Antijamming and Jamming Strategies

Competing Cognitive Radio Network (CCRN) coalesces communicator (comm) nodes and jammers to achieve maximal networking efficiency against adversarial threats. We have previously developed two contrasting approaches based on multiarmed bandit (MAB) and value-iterated Q-learning. Despite their differences, both approaches have demonstrated the efficacy of applying a machine learning technique to jointly compute comm and jammer actions in hypothetical two-network competition for an open dynamic spectrum. When sampled channel reward characteristics are time-invariant-i.e., stationarity of learned information, both MAB and Q-learning based strategies have resulted in the best possible reward empirically.

Tactical Network Integration Test Framework

Mobile Ad-Hoc Networks (MANETs) will play a significant role in future tactical military networks. These tactical networks are required to support military operations and communications on-the-move in an environment characterized by frequent changes in network topology, time varying bandwidth, interference and intermittent link blockage. The self-forming and self-healing nature of MANETs is therefore advantageous in a tactical military network.

Dynamic resource allocation for satellite communications

In large multiuser systems, such as communications satellites, the dynamic allocation of resources enables the system to simultaneously maximize usable throughput and provide acceptable communications quality to all users. In this paper the dynamic allocation problem for satellite communications is discussed. We present a multiple-rate time division multiple access (TDMA) algorithm with the ability to adapt the information transfer rate per time slot as well as time slot allocation among terminals based upon traffic loads or link conditions. Simulation results comparing this algorithm with an algorithm that does not have the ability to alter the information transfer rate algorithm show significant improvement in terms of system throughput and end-to-end delay for the proposed algorithm.

Blind Signal Classification via Sparse Coding

Abstract-We propose a novel RF signal classification method based on sparse coding, an unsupervised learning method popular in computer vision. In particular, we employ a convolutional sparse coder that can extract high-level features of an unknown received signal by maximal similarity matching against an overcomplete dictionary of filter patterns. Such dictionary can be either generated or learned in an unsupervised fashion from measured signal examples conveying no ground-truth labels. The computed sparse code is then applied to train SVM classifiers for discriminating RF signals. As a result, the proposed approach can achieve blind signal classification that requires no prior knowledge (e.g., MCS, pulse shaping) about the signals present in an arbitrary RF channel. Since modulated RF signals undergo pulse shaping to aid the matched filter detection, our method exploits variability in relative similarity against the dictionary atoms as the key discriminating factor for classification. Our experimental results indicate that we can blindly separate different classes of digitally modulated signals with a 0.703 recall and 0.246 false alarm at 20 dB SNR. Provided a small labeled dataset for supervised classifier training, we could improve the classification performance to a 0.878 recall and 0.141 false alarm.