Arnak Poghosyan

Abnormality analysis of streamed log data

We examine the determination of abnormality of streamed data using the statistical structure of the meta-data associated with it. The vital need for such a subject within a heterogeneous log based environment in real-time comes from the fact that most cloud based applications will use text-based logging as a means of reporting application behavior. The sheer volume of such logs makes retrospective analysis infeasible due to large processing and storage requirements. Our approach is based on conversion of the original data stream into meta-data (graph) and revealing the dominating (normal) statistical patterns within it. Real-time analysis of the stream compared with the meta-data model determines the degree of anomaly of the current data. The resulting graph also reveals the fundamental structure (“behavioral footprint”) of the data beyond the sources (physical or virtual devices) and processes.

Managing Cloud Infrastructures by a Multi-layer Data Analytics

Todays IT management faces the problem of “virtualized big environments” with hundreds of thousands of objects/resources as virtual machines, hosts, clusters, etc., evolving into cloud services. Admins of those infrastructures heavily rely on smart data-agnostic approaches to get reliable and accurate information regarding any current or upcoming health deterioration, increasingly requesting more proactive solutions. We architected a multi-layer enterprise analytics that employs statistical and machine learning methods to maximally automate the data center operations for an optimal performance management. We share several experience stories on application of the developed approaches and address noise and complexity reduction requirements to increase the operational efficiency of the analytics.

Ranking and Updating Beliefs Based on User Feedback: Industrial Use Cases

Incorporation of user feedback in enterprise management products can greatly enhance our understanding of modern technology challenges and amplify the ability for those products to home in to user environments. In this paper we present an entropy-based confidence determination approach to process user feedback data (direct or indirect) to automatically rank and update the beliefs of any recommender system. Several examples of application of this method are discussed in the context of VMware products. Moreover, an optimization algorithm is demonstrated for adaptive thresholding of monitoring flows based on user ratings of generated alerts effectiveness.