Maitreya Natu

Probe Station Placement for Robust Monitoring of Networks

We address the problem of selecting probe station locations from where probes can be sent to monitor all the nodes in the network. Probe station placement involves instrumentation overhead. Hence, the number of probe stations should be minimal to reduce the deployment cost. Also, probe station placement should be such that the network can be monitored even in the presence of failures. We present algorithms to select locations of probe stations so that the entire network can be monitored for computing various performance metrics. We aim to find a minimal set of probe station nodes so as to minimize the instrumentation overhead. The algorithm presented provides robust monitoring in presence of node failures. We then present algorithms to make the solution resilient to probe station failures, and to deal with weakly connected nodes. We provide an experimental evaluation of the proposed algorithms through simulation results.

Probe station selection algorithms for fault management in computer networks

In this paper, we address the problem of probe station selection. Probe station nodes are the nodes that are instrumented with the functionality of sending probes and analyzing probe results. The placement of probe stations affects the diagnosis capability of the probes sent by the probe stations. The probe station placement also involves the overhead of instrumentation. Thus it is important to minimize the required number of probe stations without compromising on the required diagnosis capability of the probes. In this paper, we address the problem of selection of probe stations to detect failures in the network. We present an algorithm for probe station selection using a reduction of the probe station selection problem to the Minimum Hitting Set problem. We address several issues involved while selecting probe stations such as link failures and probe station failures. We present experimental evaluation to show the effectiveness of the proposed approach.

Holistic Performance Monitoring of Hybrid Clouds: Complexities and Future Directions

Effective monitoring solutions are critical to the smooth running of enterprise systems. However, real-world constraints present several challenges in designing such solutions. With the increasing scale and complexity of todays enterprise IT systems and their increasing use for business-critical applications, traditional approaches to monitoring must be reconsidered. This article stresses the need for a paradigm-shift from manual intuition-led approaches to an automated analytics-driven approach to monitor the IT systems. The authors propose that analytics-led solutions can provide powerful levers to design monitoring and event management solutions for next-generation enterprise IT systems.

Adaptive Monitoring: Application of Probing to Adapt Passive Monitoring

Availability of good quality monitoring data is a vital need for management of today’s data centers. However, effective use of monitoring tools demands an understanding of the monitoring requirements that system administrators most often lack. Instead of a well-defined process of defining a monitoring strategy, system administrators adopt a manual and intuition-based approach. In this paper, we propose to replace the ad-hoc, manual, intuition-based approach with a more systematic, automated, and analytics-based approach for system monitoring. We propose an adaptive monitoring framework where end-to-end probing-based solutions are used to adapt the at-a-point monitoring tools. We present a systematic framework to use probes for adjusting monitoring levels. We present algorithms to select and analyze probes and to dynamically adapt the monitoring policies based on probe analysis. We demonstrate the effectiveness of the proposed solution using real-world examples as well as simulations.

Domain-Driven Data Mining for IT Infrastructure Support

Support analytics (i.e., statistical analysis, modeling and mining of customer/operations support tickets data) is important in service industries. In this paper, we adopt a domain-driven data mining approach to support analytics with a focus on IT infrastructure Support (ITIS) services. We identify specific business questions and then propose algorithms for answering them. The questions are: (1) How to reduce the overall workload? (2) How to improve efforts spent in ticket processing? (3) How to improve compliance to service level agreements? We propose novel formalizations of these notions and propose rigorous statistics-based algorithms for these questions. The approach is domain-driven in the sense that the results produced are directly usable by and easy to understand for end-users having no expertise in data-mining, do not require any experimentation and often discover novel and non-obvious answers. All this helps in better acceptance among end-users and more active use of the results produced. The algorithms have been implemented and have produced satisfactory results on more than 25 real-life ITIS datasets, one of which we use for illustration.

Performance debugging in data centers: Doing more with less

With the increasing scale and complexity of data centers, detecting and localizing performance faults in real-time has become both a pressing need and a challenge. While several approaches for performance debugging in data centers have been proposed, these techniques do not assume any constraints on the availability of operational data needed to detect and localize faults. We argue that collecting such operational data often requires significant instrumentation or intrusiveness, which is difficult to realize in production data centers. Such constraints complicate the deployment of existing techniques or limit their effectiveness in practice. In this paper, we argue that for performance debugging to become practical and effective in realworld systems, one needs to develop techniques that are “more effective” with “less instrumentation and intrusiveness”. We raise several issues and challenges in realizing this vision and present some initial ideas on addressing these challenges.

Automated debugging of SLO violations in enterprise systems

A critical business requirement of todays enterprise applications is automated debugging of violation of Service Level Objectives (SLOs). However, the increasing scale and complexity of these systems present various challenges in building such solutions. This problem becomes challenging mainly because of two reasons: (1) availability of large number of metrics that can potentially be the causes, (2) availability of large number of data-points. The existing techniques are either highly compute-intensive and thus are not viable for use on large volumes of data or compromise on accuracy. To successfully balance these two objectives simultaneously, we propose to intelligently prune the search space. We apply feature selection to remove irrelevant and redundant metrics. We then identify temporal regions of interest to narrow down the analysis to a smaller set of data-points. We present a comparative study of the proposed approach with other existing approaches through experimental evaluation.

Architecting the Database Access for a IT Infrastructure and Data Center Monitoring Tool

We present our experience in architecting the database access for a tool that analyzes IT infrastructure and data center utilization. We study solutions built on top of TSDB [2] and SQL Server. We provide a configuration based multi threaded approach in SQL Server based on Spring Batch [1]. Our architecture includes spring batch partitioning, JDBC-R bridge rewrite and optimized Transact-SQL procedures.

Mining frequent subgraphs to extract communication patterns in data-centres

In this paper, we propose to use graph-mining techniques to understand the communication pattern within a data-centre. We present techniques to identify frequently occurring sub-graphs within this temporal sequence of communication graphs. We argue that identification of such frequently occurring sub-graphs can provide many useful insights about the functioning of the system. We demonstrate how the existing frequent sub-graph discovery algorithms can be modified for the domain of communication graphs in order to provide computationally light-weight and accurate solutions. We present two algorithms for extracting frequent communication sub-graphs and present a detailed experimental evaluation to prove the correctness and efficiency of the proposed algorithms.

Varanus: More-with-less fault localization in data centers

Detecting and localizing performance faults is crucial for operating large enterprise data centers. This problem is relatively straightforward to solve if each entity (applications, servers, business processes) within the data center can be instrumented and monitored explicitly. Unfortunately, such instrument-everything approach is often not tenable because of the limits imposed by enterprises on the permissible amounts of instrumentation intrusiveness and monitoring overhead. In this paper, we address the problem of achieving high accuracy of detecting and localizing performance faults in data centers, while minimizing the required instrumentation intrusiveness and overhead. We present novel algorithms for solving three key subproblems: (1) How many monitors are required and where should they be placed within the data center? (2) Given the proposed instrumentation plan, how to detect the existence of performance faults accurately? and (3) How to localize the root-cause of the fault? We demonstrate the effectiveness of our approach for a real-world data center topology as well as through extensive simulations.