Nimrod Lilith

Dynamic channel allocation for mobile cellular traffic using reduced-state reinforcement learning

This paper presents a reduced-state reinforcement learning solution to the dynamic channel allocation problem in cellular telecommunication networks featuring mobile traffic and call handoffs. We examine the performance of table-based function representation used in conjunction with the on-policy reinforcement learning algorithm SARSA and show that the policy obtained using a reduced-state table-based technique provides an online dynamic channel allocation solution with superior performance in terms of new call and handoff blocking probability as well as significantly reduced memory requirements. The superior performance of the proposed state-reduced technique is illustrated in simulation examples.

Using Reinforcement Learning for Call Admission Control in Cellular Environments featuring Self-Similar Traffic

This paper details reinforcement learning architectures that efficiently provide the functions of dynamic channel allocation (DCA) and call admission control (CAC) for cellular telecommunications environments featuring both voice traffic and self-similar data traffic. These solutions are able to be implemented in a distributed manner using only localised environment information and without the need for any off-line training period. The performance of these reinforcement learning solutions are thoroughly examined via computer simulations and are shown to produce superior results in terms of both revenue raised and handoff blocking probability.

Modelling defence logistics networks

Military logistics concerns the activities required to support operational forces. It encompasses the storage and distribution of materiel, management of personnel and the provision of facilities and services. A desire to improve the efficiency and effectiveness of the Australian Defence Force logistics process has led to the investigation of rigorous military logistics models suitable for analysis and experimentation. Logistics networks can be viewed as distributed discrete event systems, and hence can be formalised with discrete event techniques which support concurrency. This paper presents a Coloured Petri Net (CPN) model of a military logistics system and discusses some of our experience in developing an initial model. Interesting modelling problems encountered, and their solutions and impact on CPN support tools, are discussed.

Approximate closed-form aggregation of a fork-join structure in generalised stochastic petri nets

In this paper an aggregation technique for generalised stochastic Petri nets (GSPNs) possessing synchronised parallel structures is presented. Parallel processes featuring synchronisation constraints commonly occur in fields such as product assembly and computer process communications, however their existence in closed networks severely complicates analysis. This paper details the derivation of computationally-simple closed-form expressions which permit the aggregation of a GSPN subnet featuring a fork-join structure. The aggregation expressions presented in this paper do not require the generation of the underlying continuous time Markov chain of the original net, and do not follow an iterative procedure. The resulting aggregated GSPN accurately approximates the stationary token distribution behaviour of the original net, and this is shown by the analysis of a number of example GSPNs.

Distributed reduced-state SARSA algorithm for dynamic channel allocation in cellular networks featuring traffic mobility

This paper presents a distributed reinforcement learning solution to the problem of dynamic channel allocation for cellular telecommunication networks in the presence of mobile call handoffs. The performance of various dynamic channel allocation schemes are compared via extensive computer simulations, and it is shown that a reduced-state SARSA reinforcement learning algorithm can achieve superior new call and handoff blocking probabilities. A new distributed reduced state SARSA algorithm is also developed which uses only local environment information readily available to the learning agent. By way of computer simulations, the distributed SARSA algorithm is shown to be capable of producing call blocking probabilities that are comparable to those obtained by the centralised learning agent.

Reinforcement Learning-Based Dynamic Guard Channel Scheme with Maximum Packing for Cellular Telecommunications Systems

This paper presents a distributed reinforcement learning solution to the problem of call admission control for cellular telecommunication networks in the presence of both voice traffic and self-similar data traffic, and user mobility. The developed call admission control architecture is designed to make use of only localised information, and therefore is suitable for implementation in a distributed manner. By way of computer simulations, the call admission control is shown to further improve the revenue raising capability and handoff blocking probability of the optimal maximum packing channel allocation scheme.

Reduced-State SARSA featuring extended channel reassignment for dynamic channel allocation in mobile cellular networks

This paper introduces a reinforcement learning solution to the problem of dynamic channel allocation for cellular telecommunication networks featuring either uniform or non-uniform offered traffic loads and call mobility. The performance of various dynamic channel allocation schemes are compared via extensive computer simulations, and it is shown that a reduced-state SARSA reinforcement learning algorithm can achieve superior new call and handoff blocking probabilities. A new reduced-state SARSA algorithm featuring an extended channel reassignment functionality and an initial table seeding is also demonstrated. The reduced-state SARSA incorporating the extended channel reassignment algorithm and table seeding is shown to produce superior new call and handoff blocking probabilities by way of computer simulations.

Reduced-State SARSA with Channel Reassignment for Dynamic Channel Allocation in Cellular Mobile Networks

This paper proposes a novel solution to the dynamic channel allocation problem in cellular telecommunication networks featuring user mobility and call handoffs. We investigate the performance of a number of reinforcement learning algorithms including Q-learning and SARSA, and show via simulations that a reduced-state version of SARSA incorporating a limited channel reassignment mechanism provides superior performance in terms of new call and handoff blocking probability and a significant reduction in memory requirements.