Anna Gorawska

Research on the Stream ETL Process

Continuously growing importance of information assisted by rapid development of systems that collect and process huge volumes of data has become a great problem in terms of processing and analyzing data. The response to current and future needs of market is a data warehouse assisted by process of data extraction. Mentioned stream ETL process enables loading real-time data without interrupting processing or conducting analysis that supports decision-making processes. This paper presents first implementation of the stream ETL process which origins from model and concept of a Stream Data Warehouse. In the first part of this paper the concept of the Stream Data Warehouse and its major components, including stream ETL, will be presented. The second part contains description of a developed stream ETL engine, as well as results of performed accuracy and efficiency analysis. Finally, paper concludes with description of future research issues that will be addressed in further research on the presented solution.

A Survey of Data Stream Processing Tools

In current international context boundaries set for applications are being pushed by the emergence of bursty and time-varying data streams required to be processed in near real-time. Furthermore, traditional techniques for data mining cannot be applied to data streams. Thus, stream-based applications must exhibit to excel at a plurality of requirements. According to defined rules presented in previous promulgated researches on this subject we differ stream-based applications and evaluate their aptitude to stream sources management. By this work we intend to present features and drawbacks of existing software coming from both industry and academic world, along with outlining our contribution to this field.

CUDA Powered User-Defined Types and Aggregates

This paper illustrates how CUDA can be successfully integrated into a RDBMS. It describes how heavyweight algebraic calculations can be efficiently performed by RDBMS by utilizing CUDAs programming model. We focus on the implementationof the complex user data type which is stored and managed by the RDBMS. In addition, we examine practicality of the custom aggregate functions and their performance when applied to CUDA powered user defined data types. We demonstrate this concept by using matrices and their multiplication as an example of a mathematical operation that requires great computational power which can be delivered by CUDA. This presented solution employs Microsoft SQL Server and the .NET platform.

Liquefied Petroleum Storage and Distribution Problems and Research Thesis

The greatest threat to the environment and aquatic life is an uncontrolled fuel leakage, which is also extremely hazardous to health and safety of people. Guaranteeing the reliability of a leak detection system is probably the ultimate purpose of fuel management systems. However, there are more problems that ought to be solved before or simultaneously with detecting possible outflows of fuel products. In this paper we highlight major research opportunities consistent with wetstock management and statistical inventory reconciliation. The main goal is to outline thesis on the nature and impact of numerous phenomena on the inventory reconciliation methods. Issues considered in this paper include but are not limited to sensor miscalibration, data acquisition, and transmission problems as well as leak detection from both, tanks and connected pipeline.

Evaluation and Development Perspectives of Stream Data Processing Systems

The following paper describes some common aspects of stream data processing systems. The paper consists of two main parts – first showing the short description, tests results and conclusions of an implemented system – the AGKPStream, while the second part focuses on proposed solutions, created upon experiences gained during development of mentioned system, as well as knowledge collected during learning about some concepts of a StreamAPAS system. The first discussed issue is a tuple construction – basic data representation. It concerns tuple time model, tuple schema and a tuple decorator. Afterwards, the stream query and scheduling problems are described.

Neural Networks in Petrol Station Objects Calibration

The fuel tank autocalibration problem is an important issue in managing the amount of fuel stored in the tank. Current values are calculated basing on fuel sold going out through nozzles - dispensing and fuel pumped into the tank by a tanker delivered. The difference in these values may point to different reasons - leakage, theft, or other errors. To pinpoint the cause it is important to rule out the case of wrong tank calibration, hence the tank autocalibration method is required. In this paper we present autocalibration method based on a neural networks algorithm, along with methods drawbacks and an alternative calibration method proposition.

Anomaly Detection in Data Streams: The Petrol Station Simulator

Developing anomaly detection systems requires diverse data for training and testing purposes. Real measurements are not necessarily reliable at this stage because it is almost impossible to find a diverse training set with exactly known characteristics. The petrol station simulator was designed to generate measurements that mimic real petrol station readings. The simulator produces datasets with exactly specified anomalies to be detected via anomaly detection system. The paper introduces foundations of the simulator with results. The discussion section presents future work in the area of stream data extraction and materialization in the Stream Data Warehouse.

Zero–Latency Data Warehouse System Based on Parallel Processing and Cache Module

Zero–Latency Data Warehouse (ZLDW) cannot be developed and formed on the basis of a standard ETL process, where time frames are limiting access to current data and blocking the ability to take users needs into account. Therefore, after profound analysis of this issue and ones related to workload balancing, an innovative system based on a Workload Balancing Unit (WBU) was created. In this paper we present innovative workload balancing algorithm – CTBE (Choose Transaction By Election), which allows to analyze all incoming transactions and create a schema of dependencies between them. Also, cache in the created WBU ensures ability to store information on incoming transactions and exchange messages with systems transmitting updates and users’ queries. By this work we intend to present an innovative system designed to support Zero–Latency Data Warehouse.

User Identity Unification in e-Commerce

Data mining applied to social media is gaining popularity. It is worth noticing that most e-commerce services also cause the formation of small communities not only services oriented toward socializing people. The results of their analysis are easier to implement. Besides, we can expect a better perception of the business by its own users, therefore the analysis of their behavior is justified. In the paper we introduce an algorithm which identifies particular customers among not logged or not registered users of a given e-commerce service. The identification of a customer is based on data that was given so as to accomplish selling procedure. Customers rarely use exactly the same identification data each time. In consequence, it is possible to check if customers create a group of unrelated individuals or if there are symptoms of social behavior.

Customer Unification in E-Commerce

Data mining applied to social media is gaining popularity. It is worth noticing that most e-commerce services also cause the formation of small communities not only services oriented toward socializing people. The results of their analysis are easier to implement. Besides, we can expect a better perception of the business by its own users, therefore the analysis of their behavior is justified. In the paper we introduce an algorithm which identifies particular customers among not logged or not registered users of a given e-commerce service. The identification of a customer is based on data that was given so as to accomplish selling procedure. Customers rarely use exactly the same identification data each time. In consequence, it is possible to check if customers create a group of unrelated individuals or if there are symptoms of social behavior.