Pankaj K. Garg

Progressive open source

The success of several Open Source™ software systems, e.g., Apache, Bind, Emacs, and Linux, has recently sparked interest in studying and emulating the software engineering principles underlying this innovative development and use model. Certain aspects of the Open Source development method, e.g., community building, open discussions for requirements and features, and evolvable and modular designs are having fundamental and far reaching consequences on general software engineering practices.To leverage such Open Source methods and tools, we have defined an innovative software engineering paradigm for large corporations: Progressive Open Source (POS). POS leverages the power of Open Source methods and tools for large corporations in a progressive manner: starting from completely within the corporation, to include partner businesses, and eventually complete Open Source. In this paper we present the design goals and principles for POS. We illustrate POS with two programs in HP: Corporate Source and the Collaborative Development Program (CDP). We present early results from both these programs suggesting the power and necessity of POS for all modern large corporations.

MUDABlue: an automatic categorization system for open source repositories

Open source communities typically use a software repository to archive various software projects with their source code, mailing list discussions, documentation, bug reports, and so forth. For example, SourceForge currently hosts over seventy thousand open source software systems. Because of the size of the rich information content, such repositories offer numerous opportunities for sharing information among projects. For example, one would like to know a set of projects that are related or similar to each other, so that the project groups can collaborate and share their work. With thousands of projects in typical repositories, however, manually locating related projects can be difficult. Hence, we propose MUDABlue, a tool that automatically categorizes software systems. MUDABlue has three major aspects: 1) it relies on no other information than the source code, 2) it determines category sets automatically, and 3) it allows a software system to be a member of multiple categories. MUDABlue has a Web interface to visualize determined categories, which eases browsing a software repository. We show the effectiveness of MUDABlues categorization capability by comparing its generated categories with that of some other existing research tools.

Automatic categorization algorithm for evolvable software archive

The number of software systems is increasing at a rapid rate. For example, SourceForge currently has about sixty thousand software systems registered, twenty-two thousand of which were added in the past twelve months. It is important for software evolution to search and use existing similar software systems from software archive. An evolution history of an existing similar software system is useful. We may even evolve a software system based on an existing one instead of creating it from scratch. We propose automatic software categorization algorithm to help finding similar software systems in software archive. At present, we leave open the issue about the nature of the categorization, and explore several known approaches including code clones-based similarity metric, decision trees, and latent semantic analysis. The results from applying each of the approaches gives us some insights into the problem space, and sets some directions for further work.

WebMon: A performance profiler for web transactions

We describe WebMon, a tool for correlated, transaction-oriented performance monitoring of web services. Data collected with WebMon can be analyzed from a variety of perspectives: business, client, transaction, or systems. Maintainers of web services can use such analysis to better understand and manage the performance of their services. Moreover WebMons data will enable the construction of more accurate performance prediction models for web services. Current web logging techniques create a log file per server making it difficult to correlate data from log files with respect to a given transaction. Additionally, data about the quality of service perceived by the client is missing entirely. WebMon overcomes these limitations by providing heterogenous instrumentation sensors and HTTP cookie-based correlators. In this paper, we present the design and implementation of of WebMon and our experience in applying WebMon to an HP Library web service.

The SMART approach for software process engineering

Describes a methodology for software process engineering and an environment, SMART, that supports it. SMART supports a process life-cycle that includes the modeling, analysis, and execution of software processes. SMARTs process monitoring capabilities can be used to provide feedback from the process execution to the process model. SMART represents the integration of three separately developed process mechanisms, and it uses two modeling formalisms (object-oriented data representation and imperative-style programming language) to bridge the gap between process modeling, analysis, and execution. SMART demonstrates the meta-environment concept, using a process modeling formalism as input specification to a generator that produces process-centered software engineering environments (PSEEs). Furthermore, SMART supports a team-oriented approach for process modeling, analysis, and execution.<<ETX>>

Towards generic application auto-discovery

The increasing complexity of enterprise applications, the expanding number of networked machines, and the rapid deployment of Internet-based business applications (E-commerce), emphasize the importance and value of application management. One of the main problems in current application management products is the amount of time and effort needed to install and customize them. Application auto-discovery is a key technology for solving this problem. In this paper, we present a generic approach to application auto-discovery along with some examples. Our approach is to create a model-based discovery engine that is driven by an application template model. While the application template model captures the variation from one application to another, the auto-discovery engine uses sophisticated mechanisms such as scoping to execute an invariant auto-discovery process.

Design-time simulation of a large-scale, distributed object system

We present a case study in using simulation at design time to predict the performance and scalability properties of a large-scale distributed object system. The system, called Consul, is a network management system designing to support hundreds of operators managing millions of network devices. It is essential that a system such as Consul be designed with performance and scalability in mind, but due to Consuls complexity and scale, it is hard to reason about performance and scalability using ad hoc techniques. We built a simulation of Consuls design to guide the design process by enabling performanace and scalability analysis of various design alternatives. A major challenge in doing design-time simulation is that many parameters for the simulation are based on estimates rather than measurements. We developed analysis methods that derive conclusions that are valid in the presence of estimation errors. In this article, we describe our scalability analysis method for design simulations of distributed object systems. The main idea is to use relative and comparative reasoning to analyze design alternatives and compare transaction behaviors. We demonstrate the analysis approach by describing its application to Consul.

Process programming by hindsight

Process programming refers to the activity of algorithmicly describing models of programming activities (processes). A serious limitation of process programming has been that . it is often hard to describe a programming process a priori. In this paper we present an approach to process programming which addresses this limitation. Our approach is based on the premise that process programs are easier to describe in hindsight rather than by foresight, and hence can be synthe- sized by observing and analyzing a recorded process history. In this paper we describe an adaptation of the well-known explanation-based learning algorithm to synthesize a process fragment from a process history. We demonstrate the useful- ness of the approach on a realistic example of coordinating file changes through a version control system.

Repository mining and Six Sigma for process improvement

In this paper, we propose to apply artifact mining in a global development environment to support measurement based process management and improvement, such as SEI/CMMIs GQ(I)M and Six Sigmas DMAIC. CMM has its origins in managing large software projects for the government and emphasizes achieving expected outcomes. In GQM, organizational goals are identified. The appropriate questions with corresponding measurements are defined and collected. Six Sigma has its origins in manufacturing and emphasizes reducing cost and defects. In DMAIC, a major component of a Six Sigma approach, sources of waste are identified. Then changes are made in the process to reduce effort and increase the quality of the product produced. GQM and Six Sigma are complementary. Both approaches rely heavily on the measurement of input and output metrics. Mining development artifacts can provide usable metrics for the application of DMAIC and GQM in the software domain.

Web transaction analysis and optimization (TAO)

In the TAO project we develop metrics, models, and infrastructure to effectively manage the performance of Web applications. We use WebMon, a novel instrumentation tool to obtain profile data for web interactions, from end-user and system component perspectives. Our analysis techniques help determine important classes of web users and their transactions. The analysis is embedded in visualization and optimization modules, enabling efficient reporting for system and business administrators, and automated resource scheduling and planning. In this paper we present an overview of TAO, and highlight some of its novel aspects, e.g., use of pixel-bar charts, web request classification, and integrated demand and capacity planning.