Pedro Casas

Unsupervised Network Intrusion Detection Systems: Detecting the Unknown without Knowledge

Traditional Network Intrusion Detection Systems (NIDSs) rely on either specialized signatures of previously seen attacks, or on expensive and difficult to produce labeled traffic datasets for user-profiling to hunt out network attacks. Despite being opposite in nature, both approaches share a common downside: they require the knowledge provided by an external agent, either in terms of signatures or as normal-operation profiles. In this paper we present UNIDS, an Unsupervised Network Intrusion Detection System capable of detecting unknown network attacks without using any kind of signatures, labeled traffic, or training. UNIDS uses a novel unsupervised outliers detection approach based on Sub-Space Clustering and Multiple Evidence Accumulation techniques to pin-point different kinds of network intrusions and attacks such as DoS/DDoS, probing attacks, propagation of worms, buffer overflows, illegal access to network resources, etc. We evaluate UNIDS in three different traffic datasets, including the well-known KDD99 dataset as well as real traffic traces from two operational networks. We particularly show the ability of UNIDS to detect unknown attacks, comparing its performance against traditional misuse-detection-based NIDSs. In addition, we also evidence the supremacy of our outliers detection approach with respect to different previously used unsupervised detection techniques. Finally, we show that the algorithms used by UNIDS are highly adapted for parallel computation, which permits to drastically reduce the overall analysis time of the system.

Passive YouTube QoE Monitoring for ISPs

Over the last decade, Quality of Experience (QoE) has become the guiding paradigm for enabling a more user-centric understanding of quality of communication networks and services. The intensifying competition among ISPs and the exponentially increasing traffic volumes caused by online video platforms like YouTube is forcing service providers to integrate QoE into their corporate DNA. This paper investigates the problem of YouTube QoE monitoring from an access providers perspective. To this end, we present three novel methods for in-network measurement of the QoE impairment that dominates user perception in the context of HTTP video-streaming: stalling of playback. Our evaluation results show that it is possible to detect application-level stalling events at high accuracy by using network-level passive probing only. However, only the most complex and most accurate approach can be used for QoE prediction due to the non-linear ties inherent in human quality perception.

Quality of Experience in Cloud services: Survey and measurements

Abstract Cloud-based systems are gaining enormous popularity due to a number of promised benefits, including ease of use in terms of deployment, administration and maintenance, high scalability as well as flexibility to create new services. However, as more personal and business applications migrate to the Cloud, the service quality becomes an important differentiator between providers, specially in the case of mobile operators. Quality of Experience (QoE) as perceived by the end-user has therefore the potential to become the guiding paradigm for managing quality provisioning and applications’ design in the Cloud. This paper presents the results of several Cloud QoE studies performed for different Cloud-based services, ranging from services with low requirements in terms of latency and interactivity (e.g., Cloud storage systems), multimedia On-Demand services (e.g., YouTube video streaming), communication and telepresence (e.g., Lync Online videoconferencing) to highly interactive services (e.g., Virtual Cloud Desktop). The results of these studies provide a ground truth basis for developing future Cloud services with QoE requirements, as well as for dimensioning the underlying network provisioning infrastructures, particularly with regard to mobile access technologies.

YOUQMON: a system for on-line monitoring of YouTube QoE in operational 3G networks

YouTube is changing the way operators manage network performance monitoring. In this paper we introduce YOUQMON, a novel on-line monitoring system for assessing the Quality of Experience (QoE) undergone by HSPA/3G customers watching YouTube videos, using network-layer measurements only. YOUQMON combines passive traffic analysis techniques to detect stalling events in YouTube video streams, with a QoE model to map stallings into a Mean Opinion Score reflecting the end-user experience. We evaluate the stalling detection performance of YOUQMON with hundreds of YouTube video streams, and present results showing the feasibility of performing real-time YouTube QoE monitoring in an operational mobile broadband network.

YouTube & Facebook Quality of Experience in mobile broadband networks

YouTube and Facebook are two of the most consumed applications in todays Internet. Together they account for more than 30% of the overall Internets traffic volume and span more than half of the Internet users worldwide. Such popularity has attracted a growing interest from the research community, particularly regarding content characterization, network performance, and privacy issues in these applications. Their extensive and ever-growing usage in cellular networks has also captured the attention of mobile operators, who need to engineer their systems to handle the load while providing good quality levels. In this paper we take a user-centric approach for YouTube and Facebook performance evaluation, analyzing the Quality of Experience (QoE) as assessed by a group of 33 mobile broadband users in a field trial. Spanning an evaluation period of 31 days and using their own 3.5G-connected laptops, users regularly reported their perceived experience on surfing their preferred YouTube and Facebook contents under changing network conditions, artificially modified through traffic shaping. Our approach is holistic and considers a three-layered evaluation methodology, including control and monitoring of the network-layer QoS, content monitoring at the network and application layers, and QoE assessment at the user-layer. To the best of our knowledge, this paper presents the first results on the evaluation of different 3.5G network conditions on YouTube and Facebook from the enduser perspective, considering everyday life Web usage scenarios.

YoMoApp: A tool for analyzing QoE of YouTube HTTP adaptive streaming in mobile networks

The performance of YouTube in mobile networks is crucial to network operators, who try to find a trade-off between cost-efficient handling of the huge traffic amounts and high perceived end-user Quality of Experience (QoE). This paper introduces YoMoApp (YouTube Performance Monitoring Application), an Android application, which passively monitors key performance indicators (KPIs) of YouTube adaptive video streaming on end-user smartphones. The monitored KPIs (i.e., player state/events, buffer, and video quality level) can be used to analyze the QoE of mobile YouTube video sessions. YoMoApp is a valuable tool to assess the performance of mobile networks with respect to YouTube traffic, as well as to develop optimizations and QoE models for mobile HTTP adaptive streaming. We test YoMoApp through real subjective QoE tests showing that the tool is accurate to capture the experience of end-users watching YouTube on smartphones.

Large-scale network traffic monitoring with DBStream, a system for rolling big data analysis

The complexity of the Internet has rapidly increased, making it more important and challenging to design scalable network monitoring tools. Network monitoring typically requires rolling data analysis, i.e., continuously and incrementally updating (rolling-over) various reports and statistics over highvolume data streams. In this paper, we describe DBStream, which is an SQL-based system that explicitly supports incremental queries for rolling data analysis. We also present a performance comparison of DBStream with a parallel data processing engine (Spark), showing that, in some scenarios, a single DBStream node can outperform a cluster of ten Spark nodes on rolling network monitoring workloads. Although our performance evaluation is based on network monitoring data, our results can be generalized to other Big Data problems with high volume and velocity.

Optimal volume anomaly detection and isolation in large-scale IP networks using coarse-grained measurements

Recent studies from major network technology vendors forecast the advent of the Exabyte era, a massive increase in network traffic driven by high-definition video and high-speed access technology penetration. One of the most formidable difficulties that this forthcoming scenario poses for the Internet is congestion problems due to traffic volume anomalies at the core network. In the light of this challenging near future, we develop in this work different network-wide anomaly detection and isolation algorithms to deal with volume anomalies in large-scale network traffic flows, using coarse-grained measurements as a practical constraint. These algorithms present well-established optimality properties in terms of false alarm and miss detection rate, or in terms of detection/isolation delay and false detection/isolation rate, a feature absent in previous works. This represents a paramount advantage with respect to current in-house methods, as it allows to generalize results independently of particular evaluations. The detection and isolation algorithms are based on a novel linear, parsimonious, and non-data-driven spatial model for a large-scale network traffic matrix. This model allows detecting and isolating anomalies in the Origin-Destination traffic flows from aggregated measurements, reducing the overhead and avoiding the challenges of direct flow measurement. Our proposals are analyzed and validated using real traffic and network topologies from three different large-scale IP backbone networks.

UNADA: unsupervised network anomaly detection using sub-space outliers ranking

Current network monitoring systems rely strongly on signature-based and supervised-learning-based detection methods to hunt out network attacks and anomalies. Despite being opposite in nature, both approaches share a common downside: they require the knowledge provided by an expert system, either in terms of anomaly signatures, or as normal-operation profiles. In a diametrically opposite perspective we introduce UNADA, an Unsupervised Network Anomaly Detection Algorithm for knowledge-independent detection of anomalous traffic. UNADA uses a novel clustering technique based on Sub-Space-Density clustering to identify clusters and outliers in multiple low-dimensional spaces. The evidence of traffic structure provided by these multiple clusterings is then combined to produce an abnormality ranking of traffic flows, using a correlation-distance-based approach. We evaluate the ability of UNADA to discover network attacks in real traffic without relying on signatures, learning, or labeled traffic. Additionally, we compare its performance against previous unsupervised detection methods using traffic from two different networks.

Modeling the YouTube stack

YouTube is one of the most popular and volume-dominant services in todays Internet, and has changed the web for ever. Consequently, network operators are forced to consider it in the design, deployment, and optimization of their networks. Taming YouTube requires a good understanding of the complete YouTube stack, from the network streaming service to the application itself. Understanding the interplays between individual YouTube functionalities and their implications for traffic and user Quality of Experience (QoE) becomes paramount nowadays. In this paper we characterize and model the YouTube stack at different layers, going from the generated network traffic to the QoE perceived by the users watching YouTube videos. Firstly, we present a network traffic model for the YouTube flow control mechanism, which permits to understand how YouTube provisions video traffic flows to users. Secondly, we investigate how traffic is consumed at the client side, deriving a simple model for the YouTube application. Thirdly, we analyze the implications for the end user, and present a model for the quality as perceived by them. This model is finally integrated into a system for real time QoE-based YouTube monitoring, highly useful to operators to assess the performance of their networks for provisioning YouTube videos. The central parameter for all the presented models is the buffer level at the YouTube application layer. This paper provides an extensive compendium of objective tools and models for network operators to better understand the YouTube traffic in their networks, to predict the playback behavior of the video player, and to assess how well they are doing in practice in delivering YouTube videos to their customers.