Stephen S. Mwanje

A Q-Learning strategy for LTE mobility Load Balancing

Cellular radio networks are seldom uniformly loaded. This motivates the need for Load Balancing (LB), as has been defined in the LTE Self-Organization standard. It is expected that on overload, a serving cell (S-cell) initiates LB to transfer some of its edge users to its neighbor cells so called target cells, by adjusting the Cell Individual Offset (CIO) parameter. In this work, we have proposed a reactive LB algorithm that adjusts the CIOs between the S-cell and all its neighbors by a fixed step φ. Our results show that the best φ depends on the load conditions in both the S-cell and its neighbors as well as on the S-cells user distribution. We then propose a Q-Learning (QL) algorithm that learns the best φ values to apply for different load conditions and demonstrate that the QL based algorithm performs better than the best fixed φ algorithm in virtually all scenarios.

Cognitive Cellular Networks: A Q-Learning Framework for Self-Organizing Networks

Self-organizing networks (SON) aim at simplifying network management (NM) and optimizing network capital and operational expenditure through automation. Most SON functions (SFs) are rule-based control structures, which evaluate metrics and decide actions based on a set of rules. These rigid structures are, however, very complex to design since rules must be derived for each SF in each possible scenario. In practice, rules only support generic behavior, which cannot respond to the specific scenarios in each network or cell. Moreover, SON coordination becomes very complicated with such varied control structures. In this paper, we propose to advance SON toward cognitive cellular networks (CCN) by adding cognition that enables the SFs to independently learn the required optimal configurations. We propose a generalized Q-learning framework for the CCN functions and show how the framework fits to a general SF control loop. We then apply this framework to two functions on mobility robustness optimization (MRO) and mobility load balancing (MLB). Our results show that the MRO function learns to optimize handover performance while the MLB function learns to distribute instantaneous load among cells.

Distributed cooperative Q-learning for mobility-sensitive handover optimization in LTE SON

Optimal settings for Handover parameters (Hysteresis and Time-to-Trigger) depend on user velocities in the network. The Self-Organization Networks (SON) standard defines the Mobility Robustness Optimization (MRO) use case for the autonomous methods of configuring the parameters in congruence to the mobility pattern. State of the art MRO solutions have relied on expert knowledge, rule based algorithms to search the parameter space; yet it is unwieldy to design rules for all possible mobility patterns in any network. In this work, we present a Q-learning MRO solution, QMRO, which learns the required parameter values appropriate for specific velocity conditions in the individual cells. We compare QMRO against the best static reference configuration (Ref) that is obtained by sweeping the parameter space. Our results show that QMRO is able to learn parameter settings that achieve similar performance to Ref in a realistic network environment where users have dynamically varying velocities.

Self-Organized handover parameter configuration for LTE

Two main parameters influence the performance of Handover (HO) in Cellular systems, the Hysteresis (Hys) and the Time-To-Trigger (TTT). In Self-Organized Networks (SON) as has been proposed for LTE, the SON methods configure and optimize these two parameters to optimize HO performance. In this paper, we model the LTE HO procedure and propose an algorithm for self-configuration of the two HO trigger parameters. We model HO performance in terms of HO rate, the Ping-Pong rate, and the Radio Link Failure rate. We observe that the optimum trigger combination can occur anywhere within the parameter space depending on the relative importance given to each of the performance metrics as implemented using a weighting function. This motivates our HO configuration algorithm that searches for optimum parameter values by undertaking optimization one parameter at a time.

An improved anomaly detection in mobile networks by using incremental time-aware clustering

With the increase of the mobile network complexity, minimizing the level of human intervention in the network management and troubleshooting has become a crucial factor. This paper focuses on enhancing the level of automation in the network management by dynamically learning the mobile network cell states and improving the anomaly detection on the individual cell level taking into consideration not just the multidimensionality of cell performance indicators, but also the sequence of cell states that have been traversed over time. Our evaluation based on the real network data shows very good performance of such a learning model being able to capture the cell behavior in time and multidimensional space. Such knowledge can improve the detection of different types of anomalies in cell functionality and enhance the process of cell failure mitigation.

Minimizing Handover Performance Degradation Due to LTE Self Organized Mobility Load Balancing

Self-Organization (SO) has been proposed to reduce capital and operational expenses as well as to improve cellular network performance. Mobility Load Balancing (MLB) and Mobility Robustness Optimization (MRO) are two of the major proposed SO use cases. Typically, MRO sets the cells Handover (HO) Hysteresis and Time to Trigger, to select the optimum point at which a HO is initiated. Conversely, MLB can be achieved by advancing HOs from overloaded to less loaded cells, commonly by adjusting the Cell Individual Offset (CIO). However, MLB affects HO metrics specifically because advancing HOs inadvertently increases Radio Link Failures (RLF) arising from overly early HOs and/or the number of HOs and Ping-Pong HOs. In this work we present a Q-Learning algorithm that learns the best MLB action to take in different load states so as to achieve a desired load transfer, but with the least effect on HO performance. The learning agent minimizes the negative HO effects by applying a penalty to MLB actions that cause high RLFs, thereby reducing the effects on HO metrics by up to 30%.

Multi-parameter Q-Learning for downlink Inter-Cell Interference Coordination in LTE SON

Inter-Cell Interference (ICI) is considered as the main reason for throughput degradation in cellular systems, especially for users at the cell edges. To mitigate ICI, ICI Coordination (ICIC) has been proposed in the context of Self Organization Networks (SON). In this paper, we present a Qlearning based ICIC algorithm, called Q-ICIC that learns for each cell the best configuration for two control parameters - the sub-band power factor and the edge-to-center boundary (ECB) so as to minimize ICI. We validate the algorithm with LTE system level simulations and show that Q-ICIC achieves considerable improvement in system performance in terms of Signal to Interference plus Noise Ratio (SINR), without compromising the coverage in the cells.

A Scoring Method for the Verification of Configuration Changes in Self-Organizing Networks

In today’s mobile communication networks the increasing reliance on Self-Organizing Network(SON) features to perform the correct optimization tasks adds a new set of challenges. In a SON-enabled network, the impact of each function’s action on the environment depends upon the actions of other functions as well. Therefore, the concept of pre-action coordination has been introduced to detect and resolve known conflicts between SON function instances. Furthermore, the idea of post-action SON verification has been proposed which is often understood as a special type of anomaly detection. It computes statistical measures on performance indicators at a relevant spatial and temporal aggregation level to assess the impact of a set of (SON-evoked) Configuration Management (CM) changes.

Fluid capacity for energy saving management in multi-layer ultra-dense 4G/5G cellular networks

There is a major demand for reducing energy consumption in mobile networks and it is expected become even more vital in the future (5G) multi-layer Ultra Dense Networks (UDNs), in which the number and density of cells in the different layers will grow dramatically. In these networks, multiple geographically overlapping layers are deployed to increase the capacity and throughput, but also increasing the energy consumption. In this paper we present an end-to-end solution that manages energy saving mechanisms in order to scale the provided capacity to the traffic. Assuming a Heterogeneous Network (HetNet) deployment, the solution dynamically selects cells to activate and/or deactivate considering the prevailing network load and the expected spectral efficiency of those cells. Evaluation in a small HetNet scenario showed that the proposed solution is able to reduce the energy consumption by more than 30%.

Concurrent cooperative games for coordinating SON functions in cognitive cellular networks

Multiple Self-Organizing Networks (SON) functions have been deveoped towards the SON promise of automating cellular network operations. Meanwhile, advancing SON towards Cognitive Cellular Networks requires the (SON) Functions (SFs) to autonomously learn the required optimal configurations. Since the SFs adjust the same or related network parameters, conflicts are bound to occur. Mechanisms that are better than current SON coordination approaches must thus be devised to manage the conflicts. In this paper we propose multi-agent Concurrent Cooperative Games (CCG) an approach where peer SFs communicate with one another so as to learn to minimize the conflicts. Using two Q-learning based SFs, we evaluate the benefits of CCG comparing against the independent functions and their uncoordinated operation. Our results show that CCG achieves good compromise especially where concurrent action among neighbor cells is avoided.