Jason Liu

Experimental evaluation of wireless simulation assumptions

All analytical and simulation research on ad~hoc wireless networks must necessarily model radio propagation using simplifying assumptions. We provide a comprehensive review of six assumptions that are still part of many ad hoc network simulation studies, despite increasing awareness of the need to represent more realistic features, including hills, obstacles, link asymmetries, and unpredictable fading. We use an extensive set of measurements from a large outdoor routing experiment to demonstrate the weakness of these assumptions, and show how these assumptions cause simulation results to differ significantly from experimental results. We close with a series of recommendations for researchers, whether they develop protocols, analytic models, or simulators for ad~hoc wireless networks.

Experimental Evaluation of Wireless Simulation Assumptions

All analytical and simulation research on ad hoc wireless networks must necessarily model radio propagation using simplifying assumptions. A growing body of research, however, indicates that the behavior of the protocol stack may depend significantly on these underlying assumptions. The standard response to this problem is a call for more realism in designing radio models. But how much realism is enough? This study is the first to approach this question by validating simulator performance (both at the physical and application layers) with the results of real-world data. Referencing an eXtensive set of measurements from a large outdoor routing eXperiment, we start by evaluating the relative realism of common assumptions made in radio model design, identifying those which provide a reasonable approXimation of reality. Although several such investigations have been made for static sensor networks, radio behavior in mobile network deployments is a much less-studied topic. We then reproduce our eXperimental setup in our simulator, and generate the same application-layer metrics under progressively smaller sets of these assumptions. By comparing the simulated outcome to the outcome of our eXperiment, we are able to discern at what point our balance of simplification and realism captures the real behavior of our target environment.

Simulation validation using direct execution of wireless Ad-Hoc routing protocols

Computer simulation is the most common approach to studying wireless ad-hoc routing algorithms. The results, however, are only as good as the models the simulation uses. One should not underestimate the importance of validation, as inaccurate models can lead to wrong conclusions. In this paper, we use direct-execution simulation to validate radio models used by ad-hoc routing protocols, against real-world experiments. This paper documents a common testbed that supports direct execution of a set of ad-hoc routing protocol implementations in a wireless network simulator. The testbed reads traces generated from real experiments, and uses them to drive direct-execution implementations of the routing protocols. Doing so we reproduce the same network conditions as in real experiments. By comparing routing behavior measured in real experiments with behavior computed by the simulation, we are able to validate the models of radio behavior upon which protocol behavior depends. We conclude that it is possible to have fairly accurate results using a simple radio model, but the routing behavior is quite sensitive to one of this models parameters. The implication is that one should: i) use a more complex radio model that explicitly models point-to-point path loss; or ii) use measurements from an environment typical of the one of interest; or iii) study behavior over a range of environments to identify sensitivities.

Composite synchronization in parallel discrete-event simulation

This paper considers a technique for composing global (barrier-style) and local (channel scanning) synchronization protocols within a single parallel discrete-event simulation. Composition is attractive because it allows one to tailor the synchronization mechanism to the model being simulated. We first motivate the problem by showing the large performance gap that can be introduced by a mismatch of model and synchronization method. Our solution calls for each channel between submodels to be classified as synchronous or asynchronous. We mathematically formulate the problem of optimally classifying channels and show that, in principle, the optimal classification can be obtained in time proportional to max{C/spl times/log C, V/spl times/N}, where C is the number of channels, V the number of unique minimal delays on those channels, and N is the number of submodels. We then demonstrate an implementation which finds an optimal solution at runtime and consider its performance on network topologies, including one of the global Internet at the autonomous system level. We find that the automated method effectively determines channel assignments that maximize performance.

RINSE: The Real-Time Immersive Network Simulation Environment for Network Security Exercises

The RINSE simulator is being developed to support large-scale network security preparedness and training exercises, involving hundreds of players and a modeled network composed of hundreds of LANs. The simulator must be able to present a realistic rendering of network behavior as attacks are launched and players diagnose events and try counter measures to keep network services operating. We describe the architecture and function of RINSE and outline how techniques like multiresolution traffic modeling and new routing simulation methods are used to address the scalability challenges of this application. We also describe in more detail new work on CPU/memory models necessary for the exercise scenarios and a latency absorption technique that help when extending the range of client tools usable by the players.

Joint Host-Network Optimization for Energy-Efficient Data Center Networking

Data centers consume significant amounts of energy. As severs become more energy efficient with various energy saving techniques, the data center network (DCN) has been accounting for 20% or more of the energy consumed by the entire data center. While DCNs are typically provisioned with full bisection bandwidth, DCN traffic demonstrates fluctuating patterns. The objective of this work is to improve the energy efficiency of DCNs during off-peak traffic time by powering off idle devices. Although there exist a number of energy optimization solutions for DCNs, they consider only either the hosts or network, but not both. In this paper, we propose a joint optimization scheme that simultaneously optimizes virtual machine (VM) placement and network flow routing to maximize energy savings, and we also build an OpenFlow based prototype to experimentally demonstrate the effectiveness of our design. First, we formulate the joint optimization problem as an integer linear program, but it is not a practical solution due to high complexity. To practically and effectively combine host and network based optimization, we present a unified representation method that converts the VM placement problem to a routing problem. In addition, to accelerate processing the large number of servers and an even larger number of VMs, we describe a parallelization approach that divides the DCN into clusters for parallel processing. Further, to quickly find efficient paths for flows, we propose a fast topology oriented multipath routing algorithm that uses depth-first search to quickly traverse between hierarchical switch layers and uses the best-fit criterion to maximize flow consolidation. Finally, we have conducted extensive simulations and experiments to compare our design with existing ones. The simulation and experiment results fully demonstrate that our design outperforms existing hostor network-only optimization solutions, and well approximates the ideal linear program.

Performance prediction of a parallel simulator

There are at least three major obstacles thwarting widespread adoption of parallel discrete-event simulation: lack of need; lack of tools; and lack of predictability in behavior and performance. The plain truth is that most simulation studies can be adequately done on ordinary serial computers. Parallel simulation tools are products of research efforts, and simply do not stand up to the demands of modern software engineering. The results of 20 years of research in parallel simulation reveal it to be a highly complex endeavour, with performance results very much dependent on implementation details and model characteristics. The Scalable Simulation Framework (SSF) is an effort to address some these concerns. It addresses lack of need in two ways; it provides a modeling API that is attractive both for serial and parallel simulation, with parallel execution requiring no change to the model, and it targets large-scale telecommunication system modeling, an application area that requires the computational capabilities of parallelism. We address the concern over unpredictable behavior. We show how we measured the internal overheads of the Dartmouth implementation of the SSF API (DaSSF), and how those measurements can be used to predict the performance of a given model, using given features of the simulator, without having to run, or even build, the model.

Empirical Validation of Wireless Models in Simulations of Ad Hoc Routing Protocols

Computer simulation has been used extensively as an effective tool in the design and evaluation of systems. One should not, however, underestimate the importance of validation—the process of ensuring whether a simulation model is an appropriate representation of the real-world system. Validation of wireless network simulations is difficult due to strong interdependencies among protocols at different layers and uncertainty in the wireless environment. The authors present an approach of coupling direct-execution simulation and traces from real outdoor experiments to validating simple wireless models that are used commonly in simulations of wireless ad hoc networks. This article documents a common testbed that supports direct execution of a set of ad hoc routing protocol implementations in a wireless network simulator. By comparing routing behavior measured in the real experiment with behavior computed by the simulation, the authors validate the models of radio behavior upon which protocol behavior depends.

An Open and Scalable Emulation Infrastructure for Large-Scale Real-Time Network Simulations

We present a software infrastructure that embeds physical hosts in a simulated network. Aiming to create a large-scale real-time virtual network testbed, our real-time interactive simulation approach combines the advantages of both simulation and emulation, by maintaining flexibility of the simulation models and increasing fidelity as real systems are included in the simulation. In our approach, real-world distributed applications and network services can run together with the real-time simulator; real packets are injected into the simulation and subject to the simulated network conditions computed as a result of both real and virtual traffic competing for network resources. A prototype of the proposed emulation infrastructure has been implemented based on virtual private network (VPN). One distinct advantage of our approach is that it does not require special hardware. Furthermore, it is flexible, secure, and scalable-attributes inherited directly from the VPN implementation. We conducted a set of preliminary experiments to assess the performance limitations of our emulation infrastructure. We also present an interesting case study to demonstrate the capability of our approach.

SVEET! a scalable virtualized evaluation environment for TCP

The ability to establish an objective comparison between high-performance TCP variants under diverse networking conditions and to obtain a quantitative assessment of their impact on the global network traffic is essential to a community-wide understanding of various design approaches. Small-scale experiments are insufficient for a comprehensive study of these TCP variants. We propose a TCP performance evaluation testbed, called SVEET, on which real implementations of the TCP variants can be accurately evaluated under diverse network configurations and workloads in large-scale network settings. This testbed combines real-time immersive simulation, emulation, machine and time virtualization techniques. We validate the testbed via extensive experiments and assess its capabilities through case studies involving real web services.