Sihyung Lee

Minerals: using data mining to detect router misconfigurations

Recent studies have shown that router misconfigurations are common and have dramatic consequences for the operations of networks. Not only can misconfigurations compromise the security of a single network, they can even cause global disruptions in Internet connectivity. Several solutions have been proposed that can detect a number of problems in real configuration files. However, these solutions share a common limitation: they are rule-based. Rules are assumed to be known beforehand, and violations of these rules are deemed misconfigurations. As policies typically differ among networks, rule-based approaches are limited in the scope of mistakes they can detect. In this paper, we address the problem of router misconfigurations using data mining. We apply association rules mining to the configuration files of routers across an administrative domain to discover local, network-specific policies. Deviations from these local policies are potential misconfigurations. We have evaluated our scheme on configuration files from a large state-wide network provider, a large university campus and a high-performance research network, and found promising results. We discovered a number of errors that were confirmed and later corrected by the network engineers. These errors would have been difficult to detect with current rule-based approaches.

Detecting network-wide and router-specific misconfigurations through data mining

Recent studies have shown that router misconfigurations are common and can have dramatic consequences to the operations of a network. Misconfigurations can compromise the security of an entire network or even cause global disruptions to Internet connectivity. Several solutions have been proposed. They can detect a number of problems in real configuration files. However, these solutions share a common limitation: they are based on rules which need to be known beforehand. Violations of these rules are deemed misconfigurations. As policies typically differ among networks, these approaches are limited in the scope of mistakes they can detect. In this paper, we address the problem of router misconfigurations using data mining. We apply association rules mining to the configuration files of routers across an administrative domain to discover local, network-specific policies. Deviations from these local policies are potential misconfigurations. We have evaluated our scheme on configuration files from a large state-wide network provider, a large university campus and a high-performance research network. In this evaluation, we focused on three aspects of the configurations: user accounts, interfaces and BGP sessions. User accounts specify the users that can access the router and define the authorized commands. Interfaces are the ports used by routers to connect to different networks. Each interface may support a number of services and run various routing protocols. BGP sessions are the connections with neighboring autonomous systems (AS). BGP sessions implement the routing policies which select the routes that are filtered and the ones that are advertised to the BGP neighbors. We included the routing policies in our study. The results are promising. We discovered a number of errors that were confirmed and corrected by the network administrators. These errors would have been difficult to detect with current predefined rule-based approaches.

Network monitoring: Present and future

Network monitoring guides network operators in understanding the current behavior of a network. Therefore, accurate and efficient monitoring is vital to ensure that the network operates according to the intended behavior and then to troubleshoot any deviations. However, the current practice of network-monitoring largely depends on manual operations, and thus enterprises spend a significant portion of their budgets on the workforce that monitor their networks. We analyze present network-monitoring technologies, identify open problems, and suggest future directions. In particular, our findings are based on two different analyses. The first analysis assesses how well present technologies integrate with the entire cycle of network-management operations: design, deployment, and monitoring. Network operators first design network configurations, given a set of requirements, then they deploy the new design, and finally they verify it by continuously monitoring the network’s behavior. One of our observations is that the efficiency of this cycle can be greatly improved by automated deployment of pre-designed configurations, in response to changes in monitored network behavior. Our second analysis focuses on network-monitoring technologies and group issues in these technologies into five categories. Such grouping leads to the identification of major problem groups in network monitoring, e.g., efficient management of increasing amounts of measurements for storage, analysis, and presentation. We argue that continuous effort is needed in improving network-monitoring since the presented problems will become even more serious in the future, as networks grow in size and carry more data.

Secure Split Assignment Trajectory Sampling: A Malicious Router Detection System

Routing infrastructure plays a vital role in the Internet, and attacks on routers can be damaging. Compromised routers can drop, modify, mis-forward or reorder valid packets. Existing proposals for secure forwarding require substantial computational overhead and additional capabilities at routers. We propose secure split assignment trajectory sampling (SATS), a system that detects malicious routers on the data plane. SATS locates a set of suspicious routers when packets do not follow their predicted paths. It works with a traffic measurement platform using packet sampling, has low overhead on routers and is applicable to high-speed networks. Different subsets of packets are sampled over different groups of routers to ensure that an attacker cannot completely evade detection. Our evaluation shows that SATS can significantly limit a malicious routers harm to a small portion of traffic in a network

Netpiler: detection of ineffective router configurations

Configuring a network is a tedious and error-prone task. In particular, configuring routing policies for a network is complex as it involves subtle dependencies in multiple routers across the network. Misconfigurations are common and certain misconfigurations can bring the Internet down. In 2005, a misconfigured router in AS 9121 blackholed traffic for tens of thousands of networks in the Internet. This paper describes NetPiler, a system that detects router misconfigurations. NetPiler consists of a routing policy configuration model and a misconfiguration detection algorithm. The model is applicable to routing policies configured on a single router as well as to network-wide configuration. Using the model, NetPiler detects configuration commands that do not influence the behavior of the network - we call these configurations ineffective commands. Although the ineffective commands could be benign, sometimes when the commands are mistakenly configured to be ineffective, they cause the network to misbehave deviating from the intended behavior. We have implemented NetPiler in approximately 128,000 lines of C++ code, and evaluated it on the configurations of four production networks. NetPiler discovers nearly a hundred ineffective commands. Some of these misconfigurations can result in loss of connectivity, access to protected networks, and financial implications by providing free transit services. We believe NetPiler can help networks to significantly reduce misconfigurations.

Improving dependability of network configuration through policy classification

As a network evolves over time, multiple operators modify its configuration, without fully considering what has previously been done. Similar policies are defined more than once, and policies that become obsolete after a transition are left in the configuration. As a result, the network configuration becomes complicated and disorganized, escalating maintenance costs and operator faults. We present a method called NetPiler, which groups common policies by discovering a set of shared features and which uses the groupings for the configuration instead of using each individual policy. Such an approach removes redundancies and simplifies the configuration while preserving the intended behavior of the configuration. We apply NetPiler to the routing policy configurations from four different networks, and reduce more than 50% of BGP communities and the related commands. In addition, we show that the reduced community definitions are sufficient to satisfy changes as the network evolves over nearly two years.

Estimation of the available bandwidth ratio of a remote link or path segments

Available bandwidth is usually sensitive to network anomalies such as physical link failure, congestion, and DDoS attack. Thus, real-time available bandwidth information can be used to detect network anomalies. Many schemes have been proposed to estimate the end-to-end available bandwidth or end-to-end capacity. However, the problem of estimating the available bandwidth for a specific remote link has not been investigated in detail yet. We propose a new scheme to estimate the available bandwidth ratio of a remote link or remote path segments, a group of consecutive links, without deploying our tool at the remote nodes. The scheme would be helpful in accurately pinpointing anomalous links. Two streams of ICMP timestamp packets are sent to both end nodes of a target link according to a Poisson process, and the available bandwidth ratio for the target link is estimated based on the measured packet delay. Since the proposed scheme needs not incur a short-term congestion, unlike conventional end-to-end available bandwidth estimation mechanisms, the intrusiveness is low and the proposed scheme overcomes the limitation of conventional approaches, inability to probe the links beyond the tight link with the minimum available bandwidth. The performance of the proposed scheme is evaluated by ns-2 simulation.

End-user perspectives of Internet connectivity problems

Network connectivity problems often create severe issues to end users, ranging from malfunction of applications (e.g., WWW and email) to complete loss of connectivity. This paper seeks to characterize these problems and discover the most efficient ways to improve network connectivity from the perspective of end users. Over a period of 7 months, we monitor network connection failures from 103 hosts used daily by end users. We find that more than 60% of downtime involves misconfigurations in the end hosts. These errors occur for various reasons, such as subtle interactions between end-host applications and routers, inconsistent network policies, and software bugs. Solving these problems can require an excessive amount of time, for expert and non-expert users alike. In contrast, problems occurring in network cores and servers are less visible to end users. For example, certain routing problems in network cores are much less likely to be seen than they are reported previously (i.e., persistent forwarding loops comprise roughly 0.02% of the observed downtime, contrary to ~2.5%, as reported in previous studies). Our results show that, although a single error in a network core or a server might affect a number of end users, the accumulated impact of errors near end hosts is much larger than that of errors in network cores and servers. We thus believe that by focusing on the problems that occur at or near end systems, we can significantly improve network availability for end users.

Correlation, visualization, and usability analysis of routing policy configurations

Network configurations implement a set of policies that control a networks behavior. Therefore, correct understanding of the configurations is vital to ensure that the network operates according to the intended policies. However, the current practice of manually reading a large number of configuration commands, which are written in low-level languages and distributed in multiple devices, is inefficient and significantly increases management costs and operator errors. We propose a system that helps decode network configurations by interpreting low-level fragmented configurations and then presenting their high-level intended policies. In particular, the proposed system is applicable to inter-domain routing policies, one of the most complex aspects of network configurations. We implement our system and evaluate its effectiveness through a set of user studies involving 44 participants. These studies examine the participants¿ comprehension of routing policies presented with our system as compared to those presented with existing configuration languages. The studies show that our system improves both accuracy, from 70% to nearly 100%, as well as time-to-task-completion, from 30 minutes to 10 minutes. We believe that our system provides a basis for a clean separation of policy intent from its implementation so that policies can be better designed and understood. We also discuss the weaknesses in usability of current network configurations and argue that all aspects of future management systems need to be designed to address these usability issues.