Mark Jacob Addleman

Towards a Training-Oriented Adaptive Decision Guidance and Support System

Information systems today have become incredibly complex and span multiple organizational networks, database and applications servers and on to the external Internet cloud resources. Consequently strategic approaches are needed to troubleshoot system failures by first identifying the component causing the failure, and thereby, further investigating the cause of the failure to solve the problem. Information regarding past troubleshooting strategies can be used to provide guidance for solving similar problems. We present a framework, DSDAware (Decision Support for Database Administrators using Warehouse-as-a-service) for developing a Decision Guidance and Support System (DGSS). The framework dynamically extracts knowledge from various correlated data sources containing systems related data and from the problem solving procedures of the human experts. The knowledge is used in a strategic problem solving approach to train new administrators by guiding them through the troubleshooting process using an interactive interface, and to offer a decision support service to the Web community. Our work specifically focuses on z/OS Mainframe DB2 database (DB) problems where the inherent complexity of the system makes troubleshooting a challenging task. The diminishing population of mainframe DB administrators (DBA) asserts the need for a DGSS for the new DBAs. The research applies text and data mining techniques for knowledge extraction, a rule-based system for knowledge representation and problem categorization, and a case-based system for providing decision support.

CAPRI: a tool for mining complex line patterns in large log data

Log files provide important information for troubleshooting complex systems. However, the structure and contents of the log data and messages vary widely. For automated processing, it is necessary to first understand the layout and the structure of the data, which becomes very challenging when a massive amount of data and messages are reported by different system components in the same log file. Existing approaches apply supervised mining techniques and return frequent patterns only for single line messages. We present CAPRI (type-CAsted Pattern and Rule mIner), which uses a novel pattern mining algorithm to efficiently mine structural line patterns from semi-structured multi-line log messages. It discovers line patterns in a type-casted format; categorizes all data lines; identifies frequent, rare and interesting line patterns, and uses unsupervised learning and incremental mining techniques. It also mines association rules to identify the contextual relationship between two successive line patterns. In addition, CAPRI lists the frequent term and value patterns given the minimum support thresholds. The line and term pattern information can be applied in the next stage to categorize and reformat multi-line data, extract variables from the messages, and discover further correlation among messages for troubleshooting complex systems. To evaluate our approach, we present a comparative study of our tool against some of the existing popular open-source research tools using three different layouts of log data including a complex multi-line log file from the z/OS mainframe system.