Anu Mary Chacko

Provenance-Aware NoSQL Databases

NoSQL stores are very widely used for BigData Analytics. These stores are built with inherent scalability and fault tolerance. But there are not much mechanism to provide security guarantees like integrity and auditability. Provenance is a metadata which captures the details of how the data reached its current state. By way of capturing provenance it is possible to enhance the functionality of NoSQL stores to verify the integrity of results. This paper presents an approach to capture provenance of NoSQL databases using logs generated by the database. A proof of concept was implemented in MongoDB and examples are used to illustrate the use of ‘Why provenance’ and ‘How-provenance’ captured.

Anomaly Detection in MapReduce Using Transformation Provenance

Data provenance is the metadata that captures information about data origin, how it was manipulated, and updated over time. Data provenance has great significance for big data applications as it provides mechanisms for verification of results. This paper discusses an approach to detect anomalies in Hadoop cluster/MapReduce job by reviewing the transformation provenance captured by mining the MapReduce logs. A rule-based framework is used to identify the patterns for extracting provenance information. The provenance information derived is converted into a provenance profile which is used for detecting anomalies in cluster and job execution.

Automatic Big Data Provenance Capture at Middleware Level in Advanced Big Data Frameworks

Huge amounts of data are being generated by Internet of Things (IoT) devices. Termed as Big Data, this data needs to be reliably stored, extracted, and analyzed. Capturing provenance of such data provides a mechanism to explain the result of data analytics and provides greater trustworthiness to the insights gathered from data analytics. Capturing the provenance of the data stored in NoSQL databases can help to understand how the data reached its current state. A holistic explanation of the results of data analytics can be achieved through the combination of provenance information of the data with results of analytics. This chapter explores the challenges of automatic provenance capture at the middleware level in three different contexts: in an analytics framework like MapReduce, in NoSQL data stores with MapReduce analytic framework, and in NoSQL stores with SQL front ends. The chapter also portrays how the provenance captured in the MapReduce framework is useful for improving the future executions of job reruns and anomaly detection, apart from its use in debugging.

Making an Application Provenance-Aware through UML – A General Scheme

An application is said to be provenance-aware when it monitors and captures the information regrading the activities of each and every process in that application. The provenance of a data item includes information about the processes and source data items that lead to its creation and current representation. This type of information or the metadata about the activities of each and every object in the application is very much important for the security purpose. The provenance information ensures the integrity of the data items and the objects involved in the application. Our approach enables an application to track the activities of each and every object involved in the application and captures the state changes of the objects into a permanent store. This information can later be queried by a Data Analyst whenever an attack by an intruder occurs into the application database. Majority of the provenance systems designed are domain specific. The methods available already for capturing the provenance are highly application specific. So there is a need to have a general methodology for capturing the provenance information automatically from the application while the application is under execution. In this paper, we present a general methodology for making an application provenance-aware by using the basic UML design diagrams of the application.

Generalized Architecture to Capture End to End Provenance in a System

Provenance is the metadata about the origin, context or history of data on how it came into its present state of being. Data in our context are files. The information about how the file was created and maintained can throw a lot of information on the credibility of data stored in the file. This information can be obtained by collecting the provenance of data. In this paper we propose a generalized framework to capture provenance information of files in a system and present a prototype tool which capture cut/copy/paste dependency of files in Linux X Windows as a proof of concept for our idea.