Krzysztof Pawlikowski

On credibility of simulation studies of telecommunication networks

In telecommunication networks, as in many other areas of science and engineering, the proliferation of computers as research tools has resulted in the adoption of computer simulation as the most commonly used paradigm of scientific investigations. This, together with a plethora of existing simulation languages and packages, has created a popular opinion that simulation is mainly an exercise in computer programming. In new computing environments, programming can be minimized, or even fully replaced, by the manipulation of icons (representing prebuilt programming objects containing basic functional blocks of simulated systems) on a computer monitor. One can say that we have witnessed another success of modern science and technology: the emergence of wonderful and powerful tools for exploring and predicting the behavior of such complex stochastic dynamic systems as telecommunication networks. But this enthusiasm is not shared by all researchers in this area. An opinion is spreading that one cannot rely on the majority of the published results on performance evaluation studies of telecommunication networks based on stochastic simulation, since they lack credibility. Indeed, the spread of this phenomenon is so wide that one can speak about a deep crisis of credibility. In this article this claim is supported by the results of a survey of over 2200 publications on telecommunication networks in proceedings of IEEE INFOCOM and such journals as IEEE Transactions on Communications, IEEE/ACM Transactions on Networking, and Performance Evaluation Journal. The discussion focuses on two important necessary conditions of a credible simulation study: use of appropriate pseudo-random generators of independent uniformly distributed numbers, and appropriate analysis of simulation output data. Having considered their perils and pitfalls, we formulate guidelines that, if observed, could help to ensure a basic level of credibility of simulation studies of telecommunication networks.

Steady-state simulation of queueing processes: survey of problems and solutions

For years computer-based stochastic simulation has been a commonly used tool in the performance evaluation of various systems. Unfortunately, the results of simulation studies quite often have little credibility, since they are presented without regard to their random nature and the need for proper statistical analysis of simulation output data. This paper discusses the main factors that can affect the accuracy of stochastic simulations designed to give insight into the steady-state behavior of queuing processes. The problems of correctly starting and stopping such simulation experiments to obtain the required statistical accuracy of the results are addressed. In this survey of possible solutions, the emphasis is put on possible applications in the sequential analysis of output data, which adaptively decides about continuing a simulation experiment until the required accuracy of results is reached. A suitable solution for deciding upon the starting point of a steady-state analysis and two techniques for obtaining the final simulation results to a required level of accuracy are presented, together with pseudocode implementations.

Handover in Mobile WiMAX Networks: The State of Art and Research Issues

The next-generation Wireless Metropolitan Area Networks, using the Worldwide Interoperability for Microwave Access (WiMAX) as the core technology based on the IEEE 802.16 family of standards, is evolving as a Fourth-Generation (4G) technology. With the recent introduction of mobility management frameworks in the IEEE 802.16e standard, WiMAX is now in competition with the existing and forthcoming generations of wireless technologies for providing ubiquitous computing solutions. However, the success of a good mobility framework largely depends on the capability of performing fast and seamless handovers irrespective of the deployed architectural scenario. Now that the IEEE has defined the Mobile WiMAX (IEEE 802.16e) MAC-layer handover management framework, the Network Working Group (NWG) of the WiMAX Forum is working on the development of the upper layers. However, the path to commercialization of a full-fledged WiMAX mobility framework is full of research challenges. This article focuses on potential handover-related research issues in the existing and future WiMAX mobility framework. A survey of these issues in the MAC, Network and Cross-Layer scenarios is presented along with discussion of the different solutions to those challenges. A comparative study of the proposed solutions, coupled with some insights to the relevant issues, is also included.

Distributed stochastic discrete-event simulation in parallel time streams

Quantitative stochastic simulation suffers from the fact that sound simulation studies require very long runlength to obtain the results with sufficient accuracy. We look at traditional approaches to distributed quantitative stochastic simulation and propose a new scenario, Multiple Replications in Parallel Time Streams (MRIP), that solves the problem in an efficient way. An implementation of MRIP in a simulation package AKAROA is also described. AKAROA accepts ordinary (non-parallel), simulation models and creates automatically the environment required for running MRIP on workstations of a local area network. Presented results show that MRIP offers linear speedup of simulation. Limitations of this scenario for running distributed quantitative stochastic simulation are also discussed.

Coverage of confidence intervals in sequential steady-state simulation☆

Abstract Stochastic discrete-event simulation has become one of the most-used tools for performance evaluation in science and engineering. But no innovation can replace the responsibility of simulators for obtaining credible results from their simulation experiments. In this paper we address the problem of the statistical correctness of simulation output data analysis, in the context of sequential steady-state stochastic simulation, conducted for studying long run behavior of stable systems. Such simulations are stopped as soon as the relative precision of estimates, defined as the relative half-width of confidence intervals at a specified confidence level, reaches the required level. We formulate basic rules for the proper experimental analysis of the coverage of steady-state interval estimators. Our main argument is that such an analysis should be done sequentially. The numerical results of our coverage analysis of the method of non-overlapping batch means and spectral analysis are presented, and compared with those obtained by the traditional, non-sequential approach. Two scenarios for stochastic simulation are considered: traditional sequential simulation on a single processor, and fast concurrent simulation based on multiple replications in parallel (MRIP), with multiple processors cooperating in the production of output data.

Do Not Trust All Simulation Studies of Telecommunication Networks

Since the birth of ARPANET and the first commercial applications of computer networks, through explosion of popularity of the Internet and wireless communications, we have witnessed increasing dependence of our civilization on information services of telecommunication networks. Their efficiency and reliability have become critically important for the well-being and prosperity of societies as well as for their security. In this situation, the significance of performance evaluation studies of current and future networks cannot be underestimated. Increasing complexity of networks has resulted in their performance evaluation studies being predominantly conducted by means of stochastic discrete-event simulation. This paper is focused on the issue of credibility of the final results obtained from simulation studies of telecommunication networks. Having discussed the basic conditions of credibility, we will show that, unfortunately, one cannot trust the majority of simulation results published in technical literature. We conclude with general guidelines for resolving this credibility crisis.

A Framework for Resource Allocation Strategies in Cloud Computing Environment

This paper presents a critical evaluation of current network resource allocation strategies and their possible applicability in Cloud Computing Environment which is expected to gain a prominent profile in the Future Internet. This research attempts to focus towards network awareness and consistent optimization of network resource allocation strategies and identifies the issues which need further investigation by the research community. A framework for network resource allocation in Cloud Computing, based on tailored active measurements, has also been proposed. The main conclusion is that network topology, traffic considerations, changing optimality criteria along with dynamic user requirements will play a dominant role in determining future Internet application architectures and protocols and in shaping resource allocation and management strategies in vital applications like Cloud Computing.

Utility of algebraic connectivity metric in topology design of survivable networks

In studies of survivable networks, it is important to be able to differentiate network topologies by means of a robust numerical measure that indicates the levels of immunity of these topologies to failures of their nodes and links. Ideally, such a measure should be sensitive to the existence of nodes or links which are more important than others, for example, if their failures cause the networks disintegration. In this paper, we suggest using an algebraic connectivity metric, adopted from spectral graph theory, namely the 2nd smallest eigenvalue of the Laplacian matrix of the network topology, instead of the average nodal degree that is usually used to characterize network connectivity in studies of the spare capacity allocation problem. Extensive simulation studies confirm that this metric is a more informative and more accurate parameter than the average nodal degree for characterizing network topologies in survivability studies.

Comparison of various estimators in simulated FGN

Abstract The Hurst parameter is the simplest numerical characteristic of self-similar long-range dependent stochastic processes. Such processes have been identified in many natural and man-made systems. In particular, since they were discovered in the Internet and other multimedia telecommunication networks a decade ago, they have been the subject of numerous investigations. Typical quantitative assessment of self-similarity and long-range dependency, begins with the estimation of the Hurst parameter H. There have been a number of techniques proposed for this. This paper reports results of a comparative analysis of the six most frequently used estimators of H. To set up a credible framework for this, the minimal acceptable sample size is first determined. The Hurst parameter estimators are then compared for bias and variance. Our experimental results have confirmed that the Abry–Veitch Daubechies Wavelet-Based (DWB) and the Whittle ML (Maximum Likelihood) estimators of H are the least biased. However, the latter has significantly smaller variance and can be applied to shorter data samples than the Abry–Veitch DWB estimator. On the other hand, the Abry–Veitch DWB estimator is computationally simpler and faster than the Whittle ML estimator.

Fast self-similar teletraffic generation based on FGN and wavelets

It is generally accepted that self-similar (or fractal) processes may provide better models of teletraffic in modern computer networks than Poisson processes. Thus, an important requirement for conducting simulation studies of telecommunication networks is the ability to generate long synthetic stochastic self-similar sequences. A new generator of pseudo-random self-similar sequences, based on fractional Gaussian noise (FGN) and a wavelet transform, is proposed and analysed in this paper. Specifically, this generator uses Daubechies wavelets. The motivation behind this selection of wavelets is that Daubechies wavelets lead to more accurate results by better matching the self-similar structure of long-range dependent processes, than other types of wavelets. The statistical accuracy and time required to produce sequences of a given (long) length are experimentally studied. This generator shows a high level of accuracy of the output data (in the sense of the Hurst parameter) and is fast. Its theoretical algorithmic complexity is O(n).