Nils Morozs

Aerial base stations with opportunistic links for next generation emergency communications

Rapidly deployable and reliable mission-critical communication networks are fundamental requirements to guarantee the successful operations of public safety officers during disaster recovery and crisis management preparedness. The ABSOLUTE project focused on designing, prototyping, and demonstrating a high-capacity IP mobile data network with low latency and large coverage suitable for many forms of multimedia delivery including public safety scenarios. The ABSOLUTE project combines aerial, terrestrial, and satellites communication networks for providing a robust standalone system able to deliver resilience communication systems. This article focuses on describing the main outcomes of the ABSOLUTE project in terms of network and system architecture, regulations, and implementation of aerial base stations, portable land mobile units, satellite backhauling, S-MIM satellite messaging, and multimode user equipments.

Distributed Heuristically Accelerated Q-Learning for Robust Cognitive Spectrum Management in LTE Cellular Systems

In this paper, we propose an algorithm for dynamic spectrum access (DSA) in LTE cellular systems-distributed ICIC accelerated Q-learning (DIAQ). It combines distributed reinforcement learning (RL) and standardized inter-cell interference coordination (ICIC) signalling in the LTE downlink, using the framework of heuristically accelerated RL (HARL). Furthermore, we present a novel Bayesian network based approach to theoretical analysis of RL based DSA. It explains a predicted improvement in the convergence behaviour achieved by DIAQ, compared to classical RL. The scheme is also assessed using large scale simulations of a stadium temporary event network. Compared to a typical heuristic ICIC approach, DIAQ provides significantly better quality of service and supports considerably higher network throughput densities. In addition, DIAQ dramatically improves initial performance, speeds up convergence, and improves steady state performance of a state-of-the-art distributed Q-learning algorithm, confirming the theoretical predictions. Finally, our scheme is designed to comply with the current LTE standards. Therefore, it enables easy implementation of robust distributed machine intelligence for full self-organisation in existing commercial networks.

Distributed Q-learning based dynamic spectrum management in cognitive cellular systems: Choosing the right learning rate

This paper presents the concept of the Win-or-Learn-Fast (WoLF) variable learning rate for distributed Q-learning based dynamic spectrum management algorithms. It demonstrates the importance of choosing the learning rate correctly by simulating a large scale stadium temporary event network. The results show that using the WoLF variable learning rate provides a significant improvement in quality of service, in terms of the probabilities of file blocking and interruption, over typical values of fixed learning rates. The results have also demonstrated that it is possible to provide a better and more robust quality of service using distributed Q-learning with a WoLF variable learning rate, than a spectrum sensing based opportunistic spectrum access scheme, but with no spectrum sensing involved.

Transfer Learning: A Paradigm for Dynamic Spectrum and Topology Management in Flexible Architectures

In this paper, we introduce a novel paradigm of transfer learning for spectrum and topology management in a rapidly deployable opportunistic network for the post disaster and temporary event scenarios. The network architecture is designed to be rapidly changing between different disaster phases, and highly flexible during the temporary event period. Transfer learning is developed to learn the dynamic radio environment from network topologies. This also allows previously learnt information in earlier phases of a deployment to be efficiently used to influence the learning process in later phases of a deployment. A Transfer Learning strategy is designed to change the knowledge base from the most recent phase via multi-agent coordination. We evaluate transfer learning paradigm in a small cell Terrestrial eNB architecture, integrated with Q-Learning and Linear Reinforcement Learning. It is demonstrated that transfer learning significantly improves the initial performance, the convergence speed and the steady state QoS, by exchanging topology information for resource prioritization.

Heuristically Accelerated Reinforcement Learning for Dynamic Secondary Spectrum Sharing

This paper examines how flexible cellular system architectures and efficient spectrum management techniques can be used to play a key role in accommodating the exponentially increasing demand for mobile data capacity in the near future. The efficiency of the use of radio spectrum for wireless communications can be dramatically increased by dynamic secondary spectrum sharing; an intelligent approach that allows unlicensed devices access to those parts of the spectrum that are otherwise underutilized by the incumbent users. In this paper, we propose a heuristically accelerated reinforcement learning (HARL)-based framework, designed for dynamic secondary spectrum sharing in Long Term Evolution cellular systems. It utilizes a radio environment map as external information for guiding the learning process of cognitive cellular systems. System level simulations of a stadium temporary event scenario show that the schemes based on the proposed HARL framework achieve high controllability of spectrum sharing patterns in a fully autonomous way. This results in a significant decrease in the primary system quality of service degradation due to interference from the secondary cognitive systems, compared with a state-of-the-art reinforcement learning solution and a purely heuristic typical LTE solution. The spectrum sharing patterns that emerge by using the proposed schemes also result in remarkable reliability of the cognitive eNodeB on the aerial platform. Furthermore, the novel principle and the general structure of heuristic functions proposed in the context of HARL are applicable to a wide range of self-organization problems beyond the wireless communications domain.

Distributed Q-learning based dynamic spectrum access in high capacity density cognitive cellular systems using secondary LTE spectrum sharing

In this paper a distributed Q-learning based dynamic spectrum access (DSA) algorithm is applied to a cognitive cellular system designed for providing ultra high capacity density with only secondary access to an LTE channel. Large scale simulations of a stadium temporary event scenario show that the distributed Q-learning based DSA scheme provides robust quality of service (QoS) and extremely high system throughput densities to the users of the stadium network, whilst successfully coexisting with a primary network of macro eNodeBs on the same LTE channel. It is also shown that incorporating spectrum awareness or spectrum sensing based admission control into the DSA algorithm in this scenario does not improve its performance. Therefore, distributed Q-learning based DSA is a viable and easily implementable solution for facilitating secondary LTE spectrum sharing in high capacity density cognitive cellular systems.

Cognitive spectrum management in dynamic cellular environments

This paper examines how novel cellular system architectures and intelligent spectrum management techniques can be used to play a key role in accommodating the exponentially increasing demand for mobile data capacity in the near future. A significant challenge faced by the artificial intelligence methods applied to such flexible wireless communication systems is their dynamic nature, e.g. network topologies that change over time. This paper proposes an intelligent case-based Q-learning method for dynamic spectrum access (DSA) which improves and stabilises the performance of cognitive cellular systems with dynamic topologies. The proposed approach is the combination of classical distributed Q-learning and a novel implementation of case-based reasoning which aims to facilitate a number of learning processes running in parallel. Large scale simulations of a stadium small cell network show that the proposed case-based Q-learning approach achieves a consistent improvement in the system quality of service (QoS) under dynamic and asymmetric network topology and traffic load conditions. Simulations of a secondary spectrum sharing scenario show that the cognitive cellular system that employs the proposed case-based Q-learning DSA scheme is able to accommodate a 28-fold increase in the total primary and secondary system throughput, but with no need for additional spectrum and with no degradation in the primary user QoS.

Intelligent Dynamic Spectrum Access in Cellular Systems with Asymmetric Topologies and Non-Uniform Traffic Loads

This paper assesses the robustness of the distributed reinforcement learning (RL) approach to dynamic spectrum access (DSA) in cellular systems with asymmetric topologies and non-uniform offered traffic distributions. Large scale simulations of a stadium small cell LTE network, employing a distributed Q-learning based DSA scheme, show that such asymmetries in the network environment cause no degradation of the QoS provided to any part of the network.

Intelligent Secondary LTE Spectrum Sharing in High Capacity Cognitive Cellular Systems

This paper investigates the distributed Q-learning approach to secondary LTE spectrum sharing, its autonomously emerging spectrum usage patterns, and their impact on the primary and secondary user quality of service (QoS). Large scale simulations of a stadium temporary event scenario show that it is capable of servicing a dramatic 51-fold increase in offered traffic, but with no need for additional spectrum and with no perceived degradation in the primary user QoS.

Using Bayesian networks for convergence analysis of intelligent dynamic spectrum access algorithms

In this paper we propose a novel Bayesian network based model for analysing convergence properties of reinforcement learning (RL) based dynamic spectrum access (DSA) algorithms. It uses a minimum complexity DSA problem for probabilistic analysis of the joint policy transitions of RL algorithms. A Monte Carlo simulation of a distributed Q-learning DSA algorithm shows that the proposed approach exhibits remarkable accuracy of predicting convergence behaviour of such algorithms. Furthermore, their behaviour can also be expressed in the form of an absorbing Markov chain, derived from the novel Bayesian network model. This representation enables further theoretical analysis of convergence properties of RL based DSA algorithms. The main benefit of the analysis tool presented in this paper is that it enables the design and theoretical evaluation of novel DSA schemes by extending the proposed Bayesian network model.