Stephen Brobst

An advanced I/O architecture for supporting mixed workloads in an active data warehouse environment

Active data warehouse environments for advanced decision support demand extreme service levels in the area of performance delivery. An active data warehouse environment, however, has very different I/O patterns than a purely traditional data warehouse implementation. An active data warehouse must be designed to support a mixed workload consisting of tactical and strategic decision support queries with highly variable I/O patterns. In this paper we present a solution for increasing the performance of mixed workloads through use of cylinder reads and optimized block sizes. Tradeoffs in implementation strategies and performance implications are discussed along with future areas of research.

Active data warehousing: a new breed of decision support

Summary form only given. Active data warehousing is rapidly changing the landscape for deployment of decision support solutions. The trend toward actionable business intelligence demands that capabilities for tactical and event-driven decision-making be supported in addition to traditional uses of the data warehouse for strategic decision-making. The resulting challenges to deliver extreme service levels in the areas of performance, availability, and data freshness require new methods for data warehouse construction. In this paper, the architectural requirements for an active data warehouse are described in detail. The evolutionary steps from first generation data warehouse implementations to active data warehouse deployment are provided as a means for incrementally delivering business value in the path toward advanced decision support capability. The service level requirements and technical building blocks for an active data warehouse deployment are described using specific examples. We explore the design tradeoffs and implementation techniques for active data warehouse deployment. Particular attention is paid to architectural topologies for successful implementation and the role of frameworks for enterprise application integration (EAI). Implementation of scalable solutions with capability for near real-time data acquisition and mixed workload management with aggressive service levels are discussed with real customer scenarios as mini case study examples.

Twelve Mistakes to Avoid When Constructing a Real-Time Data Warehouse

Real-time data warehousing is clearly emerging as a new breed of decision support. Providing both tactical and strategic decision support from a single, consistent repository of information has compelling advantages (Brobst, Rarey 2001). The result of such an implementation naturally encourages alignment of strategy development with execution of the strategy. However, a radical re-thinking of existing data warehouse architectures will need to be undertaken in many cases. Evolution toward more strict service levels in the areas of data freshness, performance, and availability are critical. Based on observations in commercial deployments of real-time data warehousing capabilities, the pages that follow identify the twelve most commonly made mistakes when designing a real-time data warehouse and give advice to help you in avoiding these pitfalls.

Fraud Detection on Large Scale Social Networks

The incredible growth of the internet use for all kinds of businesses has generated at the same time an increase of fraudulent activities, which calls for developing new methods and tools for detecting fraud and other crimes against banks and customers. Fraud detection needs to analyze and link data, which are gathered from heterogeneous data repositories, and to address problem solving algorithms optimization and parallelization, new knowledge representation paradigms, association mechanisms for linking data, and graph analysis for clustering and partitioning. We present in this paper the motivation of our study and the first steps of the work. We will focus on the emergence of new coding models based on MapReduce and SQL extensions, and on graphs paths issues.

Data mining pest, pesticide and metrological data

Abstract Agro-Metrological data is horizontally wide (dozens of variables) and vertically deep (tens of thousands of recordings). Traditional data analysis techniques are not suitable deep for such large multivariate and apparently unstructured data sets. Thus data mining is a viable option for automated detection of hidden patterns in Agro-Metrological data. As commercially available Data Mining tools are very expensive, therefore, we have developed and used our tool employing indigenous technique of Recursive Noise Removal (RNR), based on crossing minimization paradigm. In this paper we have considered two years of pest scouting data i.e. 2001 and 2002 and analyzed the results of performing clustering exploring White Fly populations.