Gautam Kar

An active approach to characterizing dynamic dependencies for problem determination in a distributed environment

We describe a methodology for identifying and characterizing dynamic dependencies between system components in distributed application environments such as e-commerce systems. The methodology relies on active perturbation of the system to identify dependencies and the use of statistical modeling to compute dependency strengths. Unlike more traditional passive techniques, our active approach requires little initial knowledge of the implementation details of the system and has the potential to provide greater coverage and more direct evidence of causality for the dependencies it identifies. We experimentally demonstrate the efficacy of our approach by applying it to a prototypical e-commerce system based on the TPC-W Web commerce benchmark, for which the active approach correctly identifies and characterizes 41 of 42 true dependencies out of a potential space of 140 dependencies. Finally, we consider how the dependencies computed by our approach can be used to simplify and guide the task of root-cause analysis, an important part of problem determination.

Dependency Analysis in Distributed Systems using Fault Injection: Application to Problem Determination in an e-commerce Environment

──────────────────────────────────────── Distributed networked applications that are being deployed in enterprise settings, increasingly rely on a large number of heterogeneous hardware and software components for providing end-to-end services. In such settings, the issue of problem diagnosis becomes vitally important, in order to minimize system outages and improve system availability. This motivates interest in dependency characterization among the different components in distributed application environments. A promising approach for obtaining dynamic dependency information is the Active Dependency Discovery technique in which a dependency graph of e-commerce transactions on hardware and software components in the system is built by individually “perturbing” the system components during a testing phase and collecting measurements corresponding to the external behavior of the system. In this paper, we propose using fault injection as the perturbation tool for dynamic dependency discovery and problem determination. We describe a method for characterizing dependencies of transactions on the system resources in a typical e-commerce environment, and show how it can aid in problem diagnosis. The method is applied to an application server middleware platform, running end-user activity composed of TPC-W transactions. Representative fault models for such an environment, that can be used to construct the fault injection campaign, are also presented.

Classification and computation of dependencies for distributed management

This paper addresses the role of dependency analysis in distributed management. The identification of dependencies becomes increasingly important in todays networked environments because applications and services rely on a variety of supporting services which might be outsourced to a service provider. However, service dependencies are not made explicit in todays systems, thus making the task of problem determination particularly difficult. Solving this problem requires the determination and computation of the dependencies between services and applications. A key contribution of the paper is a methodology for making IP-based services and applications manageable that have not been designed to include management instrumentation (which is the case today for almost every application and service). Unlike other approaches, it is not necessary to modify the application code. Instead our approach yields a technique that enumerates the characteristics and interdependencies of applications and services, thus permitting the derivation of appropriate management information.

Managing dynamic services: a contract based approach to a conceptual architecture

This paper describes a novel contract based approach for defining, deploying, monitoring and enforcing service level agreements (SLA) in a dynamic e-business environment. The current trend in application service delivery is to move away from tightly coupled systems towards structures of loosely coupled, dynamically bound systems with long and short business relationships. It appears highly likely that the next generation of e-business systems will consist of an interconnection of services, each provided by a possibly different service provider, that are coupled together to offer an end to end service to a customer. Such an environment, which we call dynamic e-business (DeB), will be governed by dynamically negotiated electronic contracts between service providers and service customers. From a management viewpoint, this development poses new challenges, such as contract based provisioning of management systems, monitoring and violation detection of dynamically agreed upon QoS parameters, problem determination and resolution, according to the terms and conditions specified in the contract. This paper proposes a management architecture for specifying, deploying and monitoring service contracts in a DeB environment, with a view to providing a basis for SLA management.

Problem Determination Using Dependency Graphs and Run-Time Behavior Models

Key challenges in managing an I/T environment for e-business lie in the area of root cause analysis, proactive problem prediction, and automated problem remediation. Our approach as reported in this paper, utilizes two important concepts: dependency graphs and dynamic runtime performance characteristics of resources that comprise an I/T environment to design algorithms for rapid root cause identification in case of problems. In the event of a reported problem, our approach uses the dependency information and the behavior models to narrow down the root cause to a small set of resources that can be individually tested, thus facilitating quick remediation and thus leading to reduced administrative costs.

Discovering Dynamic Dependencies in Enterprise Environments for Problem Determination

In order to reduce mean time to recovery (MTTR) in heterogeneous enterprise environments it should be possible to easily and quickly determine the root cause of a problem detected at a higher level, e.g. through response time violation of a transaction category, and resolve it. Many problem determination applications use a component dependency graph to pinpoint the root cause. However, such graphs are often manually constructed. This paper introduces a simple non-intrusive technique based on mining of existing runtime monitored data, to construct a dynamic dependency graph between the components of an enterprise environment. The graph is traversed to identify nodes that are the cause of response time related problems.

Mining activity data for dynamic dependency discovery in e-business systems

The growing popularity of e-business has stimulated web sites to evolve from static content servers to complex multi-tier systems built from heterogeneous server platforms. E-businesses now spend a large fraction of their IT budgets maintaining, troubleshooting, and optimizing these web sites. It has been shown that such system management activities may be simplified or automated to various extents if a dynamic dependency graph of the system were available. Currently, all known solutions to the dynamic dependency graph extraction problem are intrusive in nature, i.e. require modifications at application or middleware level. In this paper, we describe non-intrusive techniques based on data mining, which process existing monitoring data generated by server platforms to automatically extract the system component dependency graphs in multi-tier e-business platforms, without any additional application or system modification.

Determining service dependencies in distributed systems

We describe an architecture and its implementation for retrieving and handling dependency information from various managed resources in a Web-based environment. The core of our architecture is a dependency query facility that allows the application of queries and filters to dependency models; its output is a consolidated dependency graph that can then be used as input for event correlators and various management applications to display service topologies or to perform additional problem determination tasks. The definition of an XML-based notation for specifying dependencies facilitates the sharing of information between the various components involved in the process.

Threshold management for problem determination in transaction based e-commerce systems

Managing the service level objectives (SLO) in environments that implement e-commerce systems is a challenging task. It typically involves a clear understanding of how user transactions are supported by the I/T resources that comprise the e-commerce system. This paper investigates a subset of this important management problem. Using transaction to resource dependencies, the authors show how one can experimentally calculate the extent to which supporting resources for a transaction contribute to the end-to-end SLOs for that transaction. An important aspect of this process is the classification of user transactions, based on the profile of their resource usage, enabling one to set appropriate thresholds for different classes. This approach is then used to aid the detection and remediation of application performance bottlenecks.

Managing virtual storage systems: an approach using dependency analysis

We present an approach for managing the performance of virtual storage systems by experimentally identifying the dependencies that exist between various components that comprise the system. Specifically, we show how one may profile dependencies between each logical volume exported by a storage system and components that this volume uses. To do so the technique estimates the arrival rate and size of requests issued to the internal system component as a functions of arrival rate and size of requests issued to the logical volume. The complete dependency profile of the system consists of a set of such functions for READ and WRITE operations separately and for each pair: logical volume - internal system component. The empirical technique of obtaining such profiles for typical existing storage systems is presented. We propose the use of Common Information Model (CIM) as a way to express dependency and performance information in an architecture-independent manner. The dependencies between components are computed as a fraction of bandwidth that is passed on to the sub-components. We discuss how the dependency profile of the system may be used to perform root-cause analysis and early Service Level Agreement violation notification. We also demonstrate the use of the method by applying it to a Linux system using software RAID.