Nemanja Vucevic

Reinforcement learning for joint radio resource management in LTE-UMTS scenarios

The limited availability of frequency bands and their capacity limitations, together with the constantly increasing demand for high-bit-rate services in wireless communication systems, require the use of smart radio resource management strategies to ensure that different services are provided with the required quality of service (QoS) and that the available radio resources are used efficiently. In addition, the evolution of technology toward higher spectral efficiency has led to the introduction of Orthogonal Frequency-Division Multiple Access (OFDMA) by 3GPP for use in future long-term evolution (LTE) systems. However, given the current penetration of legacy technologies such as Universal Mobile Telecommunications System (UMTS), operators will face some periods in which both Radio Access Technologies (RATs) coexist. In this context, Joint Radio Resource Management (JRRM) mechanisms are helpful because they enable complementarities between different RATs to be exploited and thus facilitate more efficient use of available radio resources. This paper proposes a novel dynamic JRRM algorithm for LTE-UMTS coexistence scenarios based on Reinforcement Learning (RL), which is considered to be a good candidate for achieving the desired degree of flexibility and adaptability in future reconfigurable networks. The proposed algorithm is evaluated in dynamic environments under different load conditions and is compared with various baseline solutions.

Quality of experience evaluation under QoS-aware mobility mechanisms

Next generation wireless networks will encompass a wide range of heterogeneous technologies in the radio access part. In such networks, the all-IP paradigm has been identified as a promising solution that will contribute benefits by providing IP-based transport through the radio and core network parts. However, this concept requires a precise management of the userpsilas mobility, especially in order to preserve userpsilas quality of service (QoS) throughout the sessionpsilas lifetime. The aim of this paper is to evaluate the quality of experience (QoE) that users perceive when the different QoS-aware mobility management strategies adopted in the AROMA project are applied. A real-time testbed that provides end-to-edge QoS in all-IP heterogeneous wireless access networks has been employed to obtain QoE results that hardly could be obtained by means of simulations.

An all-IP heterogeneous wireless testbed for RAT selection and e2e QoS evaluation

In this paper, we present an all-IP heterogeneous wireless testbed for radio access technologies (RAT) selection and end-to-end (e2e) quality of service (QoS) evaluation. Presented testbed includes three different radio access networks (RANs): UTRAN, GERAN, and WLAN; and the core network (CN) based on DiffServ technology and multiprotocol label switching (MPLS). As an example two RAT selection algorithms implemented in the testbed, network-controlled cell-breathing (NCCB) and fittingness factor are described and analyzed in more details. The paper provides also simple case study that validates and compares both mentioned algorithms in a basic heterogeneous environment using the presented testbed.

Joint radio resource management for LTE-UMTS coexistence scenarios

Constantly increasing demand for throughput and quality in wireless communication systems leads to continuous research of wise radio resource management, because of the scarce availability of frequency bands and the consequent capacity limitations. In addition, technology evolution is addressed towards spectral efficient techniques that can offer higher data rates. This is the case of OFDMA (Orthogonal Frequency-Division Multiple Access), introduced by 3GPP as the technology for future Long Term Evolution (LTE). However, given the current penetration of legacy technologies such as UMTS (Universal Mobile Telecommunications System), operators will have to deal with the coexistence of multiple Radio Access Technologies (RATs), so that the exploitation of the complementarities between technologies through Joint Radio Resource Management (JRRM) mechanisms will be needed. In this paper we propose a novel dynamic JRRM algorithm for LTE-UMTS coexistence scenarios. The proposed mechanism is based on Reinforcement Learning (RL) which is considered to be a good candidate to achieve cognition in future reconfigurable networks. The proposed solution implements autonomous RL agents in each base station which decide on the allocation of the most suitable RAT to each user. We give a detailed description of the solution and analyze the behavior under various load conditions. We also demonstrate the capability of the algorithm to adjust in dynamic scenarios.

A beyond 3G real-time testbed for an All-IP heterogeneous network

This paper describes a real-time testbed emulating an all-IP B3G (Beyond 3rd Generation) network that includes UTRAN, GERAN, and WLAN emulation and the corresponding common core network based on DiffServ (Differentiated Services) technology and MPLS (Multiprotocol Label Switching). In such a complex scenario, considering real user applications and end-to-end (e2e) QoS, it is convenient to develop emulation platforms, where algorithms and applications can be tested in realistic conditions, not achievable by means of non-real-time simulations. Presented testbed will be used to evaluate three main objectives: to test the e2e QoS experienced by a user in a heterogeneous mobile environment with IP connectivity, to test and validate specific algorithms and mechanisms, and to evaluate real implementations of some subsystems.

Real-Time emulation of heterogeneous wireless networks with end-to-edge quality of service guarantees: the AROMA testbed

This work presents and describes the real-time testbed for all-IP Beyond 3G (B3G) heterogeneous wireless networks that has been developed in the framework of the European IST AROMA project. The main objective of the AROMA testbed is to provide a highly accurate and realistic framework where the performance of algorithms, policies, protocols, services, and applications for a complete heterogeneous wireless network can be fully assessed and evaluated before bringing them to a real system. The complexity of the interaction between all-IP B3G systems and user applications, while dealing with the Quality of Service (QoS) concept, motivates the development of this kind of emulation platform where different solutions can be tested in realistic conditions that could not be achieved by means of simple offline simulations. This work provides an in-depth description of the AROMA testbed, emphasizing many interesting implementation details and lessons learned during the development of the tool that may result helpful to other researchers and system engineers in the development of similar emulation platforms. Several case studies are also presented in order to illustrate the full potential and capabilities of the presented emulation platform.

Evaluation of Perceived QoS with Multimedia Applications in a Heterogeneous Wireless Network

This paper presents a study addressed to evaluate the variation in perceived QoS experienced by a user running multimedia applications in a heterogeneous wireless network. To perform this evaluation, a real-time testbed emulating an all-IP B3G network that includes UMTS Terrestrial Radio Access Network (UTRAN), GSM/EDGE Radio Access Network (GERAN), and Wireless Local Area Network (WLAN); and the corresponding common core network (CN) based on DiffServ technology and Multiprotocol Label Switching (MPLS) has been used. Results for videostreaming and video conference applications when running under interruptions due to a vertical handover (VHO) are included in this paper, pointing out the variations in behaviour of different applications.

Integrated Resource Management in Cognitive Radio

The todays radio environment features a heterogeneous mix of radio access technologies. The technology that facilitates accessing different air interfaces at different times with the same radio equipment is known as software-defined radio (SDR). A cognitive radio system manages the operational modes of SDR equipment as a function of the radio environment. This paper introduces a novel approach to resource management in cognitive radio. We coin it integrated resource management (IRM) because it integrates the management of radio and computing resources. The corresponding system then features two cognitive cycles, the computing cycle and the radio cycle. We demonstrate the suitability of our proposal in a heterogeneous radio and computing scenario. In particular, simulation results show that two simple IRM algorithms can considerably reduce the number of lost user sessions due to inappropriate reconfiguration decisions. The paper concludes that such an integrated resource management, when embedded in a cognitive radio system, can intelligently distribute the radio and computing loads, while adapting to environmental changes.

Reinforcement Learning for Active Queue Management in Mobile All-IP Networks

In future all-IP based wireless networks, like the envisaged in the long term evolution (LTE) architectures for future systems, network providers will have to deal with large traffic volumes with different QoS requirements. In order to increase exploitation of network resources wisely, intelligent adaptive solutions for class based traffic regulation are needed. In particular, active queue management (AQM) is regarded as one of these solutions to provide low queuing delay and high throughput to flows by smart packet discarding. In this paper, we propose a novel AQM solution for future all-IP networks based on a reinforcement learning scheme that allows controlling both the queuing delay and the packet loss of the different service classes. The proposed approach is evaluated through simulations and compared against other algorithms used in the literature, like the random early detection (RED) and the drop from tail (DFT), confirming the benefits of the proposed algorithm.

Reinforcement Learning for Load Management in DiffServ-MPLS Mobile Networks

Cognitive networks are envisaged to provide op- timized resource usage in future. While heterogeneity and re- source scarcity draw research attention to the wireless part, the rest of the network (mobile backhaul) is rarely considered for these improvements. The future of next generation wireless networks is probable to be all-IP, where a common flexible infrastructure is looking for dynamic autonomous solutions that cognition may provide. This work proposes a novel solution, where the introduction of reinforcement learning over multiprotocol label switching (MPLS) in a differentiated services (DiffServ) mobile backhaul should provide autonomous network adaptation aiming at en- hanced QoS capabilities. The proposed solution enables intel- ligent traffic routing by means of distributed reinforcement learning agents that base decisions on edge-gained experience. Index Terms—all-IP, DiffServ, MPLS, QoS, reinforcement learning.