David Muelas

Multi-granular, multi-purpose and multi-Gb/s monitoring on off-the-shelf systems

SUMMARY As an attempt to make network managers’ life easier, we present M3Omon, a system architecture that helps to develop monitoring applications and perform network diagnosis. M3Omon behaves as an intermediate layer between the traffic and monitoring applications that provides advanced features, high performance and low cost. Such advanced features leverage a multi-granular and multi-purpose approach to the monitoring problem. Multi-granular monitoring provides answers to tasks that use traffic aggregates to identify an event, and requires either flow records or packet data or even both to understand it and, eventually, take convenient countermeasures. M3Omon provides a simple API to access traffic simultaneously at several different granularities, i.e. packet-level, flow-level and aggregate statistics. The multi-purposed design of M3Omon allows not only performing tasks in parallel that are specifically targeted to different traffic-related purposes (e.g. traffic classification and intrusion detection) but also sharing granularities between applications, e.g. several concurrent applications fed from flow records that are provided by M3Omon. Finally, the low-cost characteristic is brought by off-the-shelf systems (the combination of open-source software and commodity hardware) and the high performance is achieved thanks to modifications in the standard NIC driver, low-level hardware interaction, efficient memory management and programming optimization. Copyright

Multi‐granular, multi‐purpose and multi‐Gb/s monitoring on off‐the‐shelf systems

SUMMARY As an attempt to make network managers’ life easier, we present M3Omon, a system architecture that helps to develop monitoring applications and perform network diagnosis. M3Omon behaves as an intermediate layer between the traffic and monitoring applications that provides advanced features, high performance and low cost. Such advanced features leverage a multi-granular and multi-purpose approach to the monitoring problem. Multi-granular monitoring provides answers to tasks that use traffic aggregates to identify an event, and requires either flow records or packet data or even both to understand it and, eventually, take convenient countermeasures. M3Omon provides a simple API to access traffic simultaneously at several different granularities, i.e. packet-level, flow-level and aggregate statistics. The multi-purposed design of M3Omon allows not only performing tasks in parallel that are specifically targeted to different traffic-related purposes (e.g. traffic classification and intrusion detection) but also sharing granularities between applications, e.g. several concurrent applications fed from flow records that are provided by M3Omon. Finally, the low-cost characteristic is brought by off-the-shelf systems (the combination of open-source software and commodity hardware) and the high performance is achieved thanks to modifications in the standard NIC driver, low-level hardware interaction, efficient memory management and programming optimization. Copyright

Functional Data Analysis: A step forward in Network Management

Network Management tasks are currently characterized by their diversity both in terms of the situations that must be faced and the data used to reach conclusions. This complex and changing context imposes diverse needs and restrictions that must be covered by management tools in order for them to be useful. In order to face current challenges, we propose the application of Functional Data Analysis (FDA) techniques in the different functional areas of Network Management. FDA can be applied to network data compression, definition of baselines, anomaly detection, or traffic classification as well as forecasting for network dimensioning.

On the impact of TCP segmentation: Experience in VoIP monitoring

Quality of Service (QoS) and Experience (QoE) monitoring is a must during the management of services deployed over the Internet. This is particularly critical for Voice over IP (VoIP), as its degradation is immediately perceived by end users. From our experience, we highlight the impact that TCP segmentation exerts on online VoIP monitoring systems. On the one hand, it makes difficult to interpret the segmented signaling messages for application monitoring. On the other hand, complete messages do not provide information about packet level behavior, which is necessary for internetworking layer monitoring. Paradoxically, the Network Management community has not paid much attention to this fact, although it compromises several VoIP management tasks. To fill in this gap, we provide an empirical evaluation of its impact for the most popular VoIP signaling protocols using traces from enterprise networks, and present the architecture and heuristic that we are currently developing to partially solve the effect of the segmentation. Our proposal avoids the overhead of reconstructing the entire data stream and, at the same time, it enables the analysis of the packets that are actually sent. To do so, it maps TCP flags with segmented application messages, and exploits data structures that reduce latency. In this way, our solution paves the way for online monitoring tools that take into account both internetworking and application layers performance.

Application of functional feature extraction to the compression of network time series

Network management actions require the retention of data representing the temporal evolution of network state, mainly in the form of time series. Nonetheless, storing and exploiting those measurements is becoming a challenge as the production rate of such data is continuously increasing and data lasting for long time periods are used. To scale up the storage and improve both the analysis and visualization of network measurements, we apply Functional Principal Components Analysis (FPCA) to extract the most meaningful functional features for network time series, pruning those with low informational importance. We compare such algorithm with other state-of-the-art proposals, and show that it achieves lower error for the representation of atypical observations even with higher compression ratios.

Dictyogram: A statistical approach for the definition and visualization of network flow categories

Network managers have to deal with tons of measurement data provided by monitoring systems. Such data is difficult to both process and translate into concrete management actions. As an attempt to make managerial work easier, we propose a novel statistical approach that summarizes the behavior of network flow characteristics - e.g., flow sizes or durations. Bearing in mind that losses in the summarized information can lead to restricted or even erroneous conclusions, our approach solves this by exploiting the probability integral transform theorem. This theorem allows the definition of a set of intervals, mapped into concrete categories, where the number of flows according to a given characteristic would be uniformly distributed among categories. This eases the use of both statistical tests and simple visual inspection to detect changes in the behavior of the characteristic under analysis, as typically abrupt changes are understood as signs of intrusion, malfunction or other types of anomalies. This proposal gave rise to the visualization and analytical framework Dictyogram, which has been applied to monitor the Spanish Academic Network - more than one million users. Its results are shown as a case study assessing the usefulness of our proposal.

On the Use of Affordable COTS Hardware for Network Measurements: Limits and Good Practices

Wireless access technologies are widespread in domestic scenarios, and end users extensively use mobile phones or tablets to browse the Web. Therefore, methods and platforms for the measurement of network key performance indicators must be adapted to the peculiarities of this environment. In this light, the experiments should capture the true conditions of such connections, particularly in terms of the hardware and multi-device interactions that are present in real networks. On the basis of this, this paper presents an evaluation of the capabilities of several affordable commercial off-the-shelf (COTS) devices as network measuring probes, for example, computers-on-module or domestic routers with software measurement tools. Our main goal is to detect the limits of such devices and define a guide of good practices to optimize them. Hence, our work paves the way for the development of fair measurement systems in domestic networks with low expenditures. The obtained experimental results show that these types of devices are suitable as network measuring probes, if they are adequately configured and minimal accuracy losses are assumable.

Software-Driven Definition of Virtual Testbeds to Validate Emergent Network Technologies

The lack of privileged access to emergent and operational deployments is one of the key matters during validation and testing of novel telecommunication systems and technologies. This matter jeopardizes the repeatability of experiments, which results in burdens for innovation and research in these areas. In this light, we present a method and architecture to make the software-driven definition of virtual testbeds easier. As distinguishing features, our proposal can mimic operational deployments by using high-dimensional activity patterns. These activity patterns shape the effect of a control module that triggers agents for the generation of network traffic. This solution exploits the capabilities of network emulation and virtualization systems, which nowadays can be easily deployed in commodity servers. With this, we accomplish a reproducible definition of realistic experimental conditions and the introduction of real agent implementations in a cost-effective fashion. We evaluate our solution in a case study that is comprised of the validation of a network-monitoring tool for Voice over IP (VoIP) deployments. Our experimental results support the viability of the method and illustrate how this formulation can improve the experimentation in emergent technologies.

Estimation of the parameters of token-buckets in multi-hop environments

Abstract Bandwidth verification in shaping scenarios receives much attention of both operators and clients because of its impact on Quality of Service (QoS). As a result, measuring shapers’ parameters, namely the Committed Information Rate (CIR), Peak Information Rate (PIR) and Maximum Burst Size (MBS), is a relevant issue when it comes to assess QoS. In this paper, we present a novel algorithm, TBCheck, which serves to accurately measure such parameters with minimal intrusiveness. These measurements are the cornerstone for the validation of Service Level Agreements (SLA) with multiple shaping elements along an end-to-end path. As a further outcome of this measurement method, we define a formal taxonomy of multi-hop shaping scenarios. A thorough performance evaluation covering the latter taxonomy shows the advantages of TBCheck compared to other tools in the state of the art, yielding more accurate results even in the presence of cross-traffic. Additionally, our findings show that MBS estimation is unfeasible when the link load is high, regardless the measurement technique, because the token-bucket will always be empty. Consequently, we propose an estimation policy which maximizes the accuracy by measuring CIR during busy hours and PIR and MBS during off-peak hours.

Online detection of pathological TCP flows with retransmissions in high-speed networks

Abstract Online Quality of Service (QoS) assessment in high speed networks is one of the key concerns for service providers, namely to detect QoS degradation on-the-fly as soon as possible and avoid customers’ complaints. In this regard, a Key Performance Indicator (KPI) is the number of TCP retransmissions per flow, which is related to packet losses or increased network and/or client/server latency. However, to accurately detect TCP retransmissions the whole sequence number list should be tracked which is a challenging task in multi-Gb/s networks. In this paper we show that the simplest approach of counting as a retransmission a packet whose sequence number is smaller than the previous one is enough to detect pathological flows with severe retransmissions. Such a lightweight approach eliminates the need of tracking the whole TCP flow history, which severely restricts traffic analysis throughput. Our findings show that low False Positive Rates (FPR) and False Negative Rates (FNR) can be achieved in the detection of such pathological flows with severe retransmissions, which are of paramount importance for QoS monitoring. Most importantly, we show that live detection of such pathological flows at 10 Gb/s rate per processing core is feasible.