Francisco Bernardo

An Application of Reinforcement Learning for Efficient Spectrum Usage in Next-Generation Mobile Cellular Networks

This paper proposes reinforcement learning as a foundational stone of a framework for efficient spectrum usage in the context of next-generation mobile cellular networks. The objective of the framework is to efficiently use the spectrum in a cellular orthogonal frequency-division multiple access network while unnecessary spectrum is released for secondary spectrum usage within a private commons spectrum access model. Numerical results show that the proposed framework obtains the best performance compared with other approaches for spectrum assignment. Moreover, the framework is relatively simple to implement in terms of computational requirements and signaling overhead.

Self-Optimization of Spectrum Assignment and Transmission Power in OFDMA Femtocells

This paper presents a framework based on self-organization to jointly self-optimize the spectrum assignment and the transmission power in the context of downlink OFDMA femtocell deployments. It is a distributed framework where each femtocell acts as an autonomous entity. Femtocells perform spectrum and transmission power assignment based on users’ reported measurements, which are employed to sense intercell interference, including that from other femtocells or from macrocells in two-layer deployments. Results have been obtained for a realistic indoor femtocell deployment with and without macrocell interference. The paper shows that, thanks to self-organization, sensible performance improvements can be achieved in terms of spectral efficiency and power consumption reductions.

Intercell Interference Management in OFDMA Networks: A Decentralized Approach Based onReinforcement Learning

This paper presents a decentralized framework for dynamic spectrum assignment in multicell orthogonal frequency division multiple access (OFDMA) networks. The proposed framework allows each cell to autonomously decide the frequency resources it should use through a procedure that incorporates concepts from self-organization and machine learning in multiagent systems (MASs). Simulation results have been obtained for several scenarios, including both macrocells (MCs) and femtocells (FCs), revealing important improvements in terms of spectral efficiency and intercell interference mitigation over reference approaches, and close performance with the one obtained by a centralized strategy. Results also suggest that the framework would be practical for future FC cellular deployments where a high degree of independence of the network nodes is expected to reduce operational costs.

Quality of experience evaluation under QoS-aware mobility mechanisms

Next generation wireless networks will encompass a wide range of heterogeneous technologies in the radio access part. In such networks, the all-IP paradigm has been identified as a promising solution that will contribute benefits by providing IP-based transport through the radio and core network parts. However, this concept requires a precise management of the userpsilas mobility, especially in order to preserve userpsilas quality of service (QoS) throughout the sessionpsilas lifetime. The aim of this paper is to evaluate the quality of experience (QoE) that users perceive when the different QoS-aware mobility management strategies adopted in the AROMA project are applied. A real-time testbed that provides end-to-edge QoS in all-IP heterogeneous wireless access networks has been employed to obtain QoE results that hardly could be obtained by means of simulations.

Distributed spectrum management based on reinforcement learning

This paper presents a novel distributed framework to decide the spectrum assignment in a primary cellular radio access network. The distributed nature of the framework allows each cell to autonomously decide (by means of machine learning procedures) the best frequencies to use in order to maximize spectral efficiency, preserve quality-of-service, and generate spectrum gaps, so that secondary cognitive radio networks can improve overall spectrum usage. The proposed distributed framework has been validated over a downlink multicell OFDMA radio access network, showing comparable performance results with respect to its centralized counterpart and superior performance with respect to fixed frequency planning schemes.

Dynamic spectrum assignment in multicell OFDMA networks enabling a secondary spectrum usage: DSA IN MULTICELL OFDMA ENABLING SECONDARY SPECTRUM USAGE

The aim of this paper is to propose a set of novel Dynamic Spectrum Assignment (DSA) algorithms for multicell Orthogonal Frequency Division Multiple Access (OFDMA) scenarios. Thanks to the proposed algorithms, licensed spectrum holders can release spectrum bands in large geographical areas to be leased to other secondary markets in a cognitive radio environment, while preserving the quality of service (QoS) of the licensed users in the system. Results are obtained comparing the proposed schemes against other conventional frequency reuse strategies, revealing significant improvements in terms of both spectral efficiency and opportunities for secondary usage. Copyright

An all-IP heterogeneous wireless testbed for RAT selection and e2e QoS evaluation

In this paper, we present an all-IP heterogeneous wireless testbed for radio access technologies (RAT) selection and end-to-end (e2e) quality of service (QoS) evaluation. Presented testbed includes three different radio access networks (RANs): UTRAN, GERAN, and WLAN; and the core network (CN) based on DiffServ technology and multiprotocol label switching (MPLS). As an example two RAT selection algorithms implemented in the testbed, network-controlled cell-breathing (NCCB) and fittingness factor are described and analyzed in more details. The paper provides also simple case study that validates and compares both mentioned algorithms in a basic heterogeneous environment using the presented testbed.

A Self-Organized Spectrum Assignment Strategy in Next Generation OFDMA Networks Providing Secondary Spectrum Access

This paper proposes a Self-organized Spectrum Assignment strategy in the context of next generation multicell Orthogonal Frequency Division Multiple Access networks. The proposed strategy is able to dynamically find spectrum assignments per cell depending on the spatial distribution of the users over the scenario, opening new spectrum access opportunities for secondary spectrum usage. Reinforcement Learning methodology has been employed to implement the strategy, which compared with other fixed and dynamic spectrum assignment strategies shows the best tradeoff between spectral efficiency and Quality-of-Service while releases spectrum in large geographical areas.

A beyond 3G real-time testbed for an All-IP heterogeneous network

This paper describes a real-time testbed emulating an all-IP B3G (Beyond 3rd Generation) network that includes UTRAN, GERAN, and WLAN emulation and the corresponding common core network based on DiffServ (Differentiated Services) technology and MPLS (Multiprotocol Label Switching). In such a complex scenario, considering real user applications and end-to-end (e2e) QoS, it is convenient to develop emulation platforms, where algorithms and applications can be tested in realistic conditions, not achievable by means of non-real-time simulations. Presented testbed will be used to evaluate three main objectives: to test the e2e QoS experienced by a user in a heterogeneous mobile environment with IP connectivity, to test and validate specific algorithms and mechanisms, and to evaluate real implementations of some subsystems.

A Novel Framework for Dynamic Spectrum Management in MultiCell OFDMA Networks Based on Reinforcement Learning

In this work the feasibility of Reinforcement Learning (RL) for Dynamic Spectrum Management (DSM) in the context of next generation multicell Orthogonal Frequency Division Multiple Access (OFDMA) networks is studied. An RL-based algorithm is proposed and it is shown that the proposed scheme is able to dynamically find spectrum assignments per cell depending on the spatial distribution of the users over the scenario. In addition the proposed scheme is compared with other fixed and dynamic spectrum strategies showing the best tradeoff between spectral efficiency and Quality-of-Service (QoS).