Marcus Hilbrich

Automatic Analysis of Large Data Sets: A Walk-Through on Methods from Different Perspectives

Analyzing data is one of todays hot topics. A complete list of fields of research and buzzwords associated with automatic analysis would extend beyond this document. The importance of this topic stems from the amount of data currently produced in research, engineering, and other fields. The size of these data sets renders manual analysis infeasible. Automatic analysis methods are required to cope with the data sets produced. The algorithms for filtering, categorization, and analysis have a long tradition and are manifold. This raises the question for the best algorithm. The authors of this paper give an overview about manifold automatic analysis approaches along with a classification of these approaches with regard to three different fields of research.

Achieving scalability for job centric monitoring in a distributed infrastructure

Job centric monitoring allows to observe jobs on remote computing resources. It may offer visualisation of recorded monitoring data and helps to find faulty or misbehaving jobs. If installations like grids or clouds are observed monitoring data of many thousands of jobs have to be handled. The challenge of job centric monitoring infrastructures is to store, search and access data collected in huge installations like grids or clouds. We take this challenge with a distributed layer based architecture which provides a uniform view to all monitoring data. The concept of this infrastructure called SLAte and an analysis of the scalability is provided in this paper.

IDENTIFYING LIMITS OF SCALABILITY IN DISTRIBUTED, HETEROGENEOUS, LAYER BASED MONITORING CONCEPTS LIKE SLAte

In this paper we present the concept of a scalable job centric monitoring infrastructure. The overall performance of this distributed, layer based architecture called SLAte can be increased by installing additional servers to adapt to the demands of the monitored resources and users. Another important aspect is to offer a uniform global view on all data which are stored distributed to provide an easy access for users or visualisation tools. Additionally we discus the impact of these uniform access layer on scalability.

Analysis of Series of Measurements from Job-Centric Monitoring by Statistical Functions

The rising number of executed programs (jobs) enabled by the growing amount of available resources from Clouds, Grids, and HPC (for example) has resulted in an enormous number of jobs. Nowadays, most of the executed jobs are mainly unobserved, so unusual behavior, non-optimal resource usage, and silent faults are not systematically searched and analyzed. Job-centric monitoring enables permanent job observation and, thus, enables the analysis of monitoring data. In this paper, we show how statistic functions can be used to analyze job-centric monitoring data and how the methods compare to more-complex analysis methods. Additionally, we present the usefulness of job-centric monitoring based on practical experiences.

Time-Aligned Similarity Calculations for Job-Centric Monitoring

In job-centric monitoring, monitors gather series of measurements, e.g., the used CPU load, per job. In domains where jobs are expected to behave similar, job-centric monitoring allows detecting misbehaving jobs based on a reference series of measurements. However, current detection approaches neglect time-drifts in series, e.g., caused by different CPU speeds and therefore potentially cause false positives.To cope with this issue, this paper introduces a novel approach to compensate such time-drifts. Our approach is based on a transformation that aligns a series of measurements to the reference series time. In a proof-of-concept with synthetic job-centric monitoring data, we show that our approach reduces the number of false positives significant.

Distributed Accounting in Scope of Privacy Preserving

Accounting is an essential part of distributed computing infrastructures, regardless whether these are more service-driven like Clouds or more computing oriented like traditional Grid Computing environments. Those infrastructures have evolved over more than the last decade and additional. beside the further development towards service-oriented architectures, the business aspect of especially Cloud Computing solutions becomes more and more relevant. In this paper we focus on user-centric aspects like privacy preserving methods to hide the users behaviour and to collect only necessary information for billing, under the assumption that an accounting system has to be integrated in the computing infrastructure and that a central interface is still desirable for billing and financial clearing.

A Security Approach for Credential-Management in Distributed Heterogeneous Systems

In distributed computing environments information is stored in different locations using different technical systems. Challenging is to allow easy access to heterogeneous resources without an uniform user management and to combine data from different resources for further processing. To delegate the user requests the according login data for different resources have to be present in a central location. The management of these highly sensible data comprises miscellaneous security aspects, e.g. to develop methods to store login data in a secure way. In this paper a security concept to deal with sensitive login data in a central component is presented. This includes a collaboration with the operators of the distributed resources and is discussed in the context of the knowledge infrastructure WisNetGrid.