Ricardo Jorge Santos

Real-time data warehouse loading methodology

A data warehouse provides information for analytical processing, decision making and data mining tools. As the concept of real-time enterprise evolves, the synchronism between transactional data and data warehouses, statically implemented, has been redefined. Traditional data warehouse systems have static structures of their schemas and relationships between data, and therefore are not able to support any dynamics in their structure and content. Their data is only periodically updated because they are not prepared for continuous data integration. For real-time enterprises with needs in decision support purposes, real-time data warehouses seem to be very promising. In this paper we present a methodology on how to adapt data warehouse schemas and user-end OLAP queries for efficiently supporting real-time data integration. To accomplish this, we use techniques such as table structure replication and query predicate restrictions for selecting data, to enable continuously loading data in the data warehouse with minimum impact in query execution time. We demonstrate the efficiency of the method by analyzing its impact in query performance using benchmark TPC-H executing query workloads while simultaneously performing continuous data integration at various insertion time rates.

Optimizing data warehouse loading procedures for enabling useful-time data warehousing

The purpose of a data warehouse is to aid decision making. As the real-time enterprise evolves, synchronism between transactional data and data warehouses is redefined. To cope with real-time requirements, the data warehouses must be able to enable continuous data integration, in order to deal with the most recent business data. Traditional data warehouses are unable to support any dynamics in structure and content while they are available for OLAP. Their data is periodically updated because they are unprepared for continuous data integration. For real-time enterprises with needs in decision support while the transactions are occurring, (near) real-time data warehousing seem very promising. In this paper we present a survey on testing todays most used loading techniques and analyze which are the best data loading methods, presenting a methodology for efficiently supporting continuous data integration for data warehouses. To accomplish this, we use techniques such as table structure replication with minimum content and query predicate restrictions for selecting data, to enable loading data in the data warehouse continuously, with minimum impact in query execution time. We demonstrate the efficiency of the method using benchmark TPC-H and executing query workloads while simultaneously performing continuous data integration.

In Vivo Modulation of Nitric Oxide Concentration Dynamics Upon Glutamatergic Neuronal Activation in the Hippocampus

Nitric oxide (•NO) is a labile endogenous free radical produced upon glutamatergic neuronal activity in hippocampus by neuronal nitric oxide synthase (nNOS), where it acts as a modulator of both synaptic plasticity and cell death associated with neurodegeneration. The low CNS levels and fast time dynamics of this molecule require the use of rapid analytical methods that can more accurately describe its signaling in vivo. This is critical for understanding how the kinetics of •NO‐dependent signaling pathways is translated into physiological or pathological functions. In these studies, we used •NO selective microelectrodes coupled with rapid electrochemical recording techniques to characterize for the first time the concentration dynamics of •NO endogenously produced in hippocampus in vivo following activation of ionotropic glutamate receptors. Both L‐glutamate (1–100 mM) and N‐methyl‐D‐aspartate (NMDA; 0.01–5 mM) produced transient, dose‐dependent increases in extracellular •NO concentration. The production of •NO in the hippocampus by glutamate was decreased by the nNOS inhibitor 7‐NI. Intraperitoneal administration of the NMDA receptor blocker, MK‐801, and the inhibitor of α‐amino‐3‐hydroxy‐5‐methyl‐4‐isoazolepropionic acid (AMPA) receptor, NBQX, applied locally greatly attenuated glutamate‐evoked overflow of •NO. Thus, •NO overflow elicited by activation of glutamate receptors appeared to result from an integrated activation of ionotropic glutamate receptors, both of the NMDA and AMPA receptors subtypes. Additionally, distinct concentration dynamics was observed in the trisynaptic loop with stronger and longer lasting effects of glutamate activation on •NO overflow seen in the CA1 region as compared with the dentate gyrus. Overall, the results provide a quantitative and temporal basis for a better understanding of •NO activity in the rat hippocampus.

A survey on data security in data warehousing: Issues, challenges and opportunities

Data Warehouses (DWs) are the enterprises most valuable assets in what concerns critical business information, making them an appealing target for malicious inside and outside attackers. Given the volume of data and the nature of DW queries, most of the existing data security solutions for databases are inefficient, consuming too many resources and introducing too much overhead in query response time, or resulting in too many false positive alarms (i.e., incorrect detection of attacks) to be checked. In this paper, we present a survey on currently available data security techniques, focusing on specific issues and requirements concerning their use in data warehousing environments. We also point out challenges and opportunities for future research work in this field.

Approaches and Challenges in Database Intrusion Detection

Databases often support enterprise business and store its secrets. This means that securing them from data damage and information leakage is critical. In order to deal with intrusions against database systems, Database Intrusion Detection Systems (DIDS) are frequently used. This paper presents a survey on the main database intrusion detection techniques currently available and discusses the issues concerning their application at the database server layer. The identified weak spots show that most DIDS inadequately deal with many characteristics of specific database systems, such as ad hoc workloads and alert management issues in data warehousing environments, for example. Based on this analysis, research challenges are presented, and requirements and guidelines for the design of new or improved DIDS are proposed. The main finding is that the development and benchmarking of specifically tailored DIDS for the context in which they operate is a relevant issue, and remains a challenge. We trust this work provides a strong incentive to open the discussion between both the security and database research communities.

A data masking technique for data warehouses

Data Warehouses (DWs) are the enterprises most valuable asset in what concerns critical business information, making them an appealing target for attackers. Packaged database encryption solutions are considered the best solution to protect sensitive data. However, given the volume of data typically processed by DW queries, the existing encryption solutions heavily increase storage space and introduce very large overheads in query response time, due to decryption costs. In many cases, this performance degradation makes encryption unfeasible for use in DWs. In this paper we propose a transparent data masking solution for numerical values in DWs based on the mathematical modulus operator, which can be used without changing user application and DBMS source code. Our solution provides strong data security while introducing small overheads in both storage space and database performance. Several experimental evaluations using the TPC-H decision support benchmark and a real-world DW are included. The results show the overall efficiency of our proposal, demonstrating that it is a valid alternative to existing standard encryption routines for enforcing data confidentiality in DWs.

Balancing Security and Performance for Enhancing Data Privacy in Data Warehouses

Data Warehouses (DWs) store the golden nuggets of the business, which makes them an appealing target. To ensure data privacy, encryption solutions have been used and proven efficient in their security purpose. However, they introduce massive storage space and performance overheads, making them unfeasible for DWs. We propose a data masking technique for protecting sensitive business data in DWs that balances security strength with database performance, using a formula based on the mathematical modular operator. Our solution manages apparent randomness and distribution of the masked values, while introducing small storage space and query execution time overheads. It also enables a false data injection method for misleading attackers and increasing the overall security strength. It can be easily implemented in any DataBase Management System (DBMS) and transparently used, without changes to application source code. Experimental evaluations using a real-world DW and TPC-H decision support benchmark implemented in leading commercial DBMS Oracle 11g and Microsoft SQL Server 2008 demonstrate its overall effectiveness. Results show substantial savings of its implementation costs when compared with state of the art data privacy solutions provided by those DBMS and that it outperforms those solutions in both data querying and insertion of new data.

Securing data warehouses from web-based intrusions

Decision support for 24/7 enterprises requires 24/7 available Data Warehouses (DWs). In this context, web-based connections to DWs are used by business management applications demanding continuous availability. Given that DWs store highly sensitive business data, a web-based connection provides a door for outside attackers and thus, creates a main security issue. Database Intrusion Detection Systems (DIDS) deal with intrusions in databases. However, given the distinct features of DW environments most DIDS either generate too many false alarms or too low intrusion detection rates. This paper proposes a real-time DIDS explicitly tailored for web-access DWs, functioning at the SQL command level as an extension of the DataBase Management System, using an SQL-like rule set and predefined checkups on well-defined DW features, which enable wide security coverage. We also propose a risk exposure method for ranking alerts which is much more effective than alert correlation techniques.

Leveraging 24/7 Availability and Performance for Distributed Real-Time Data Warehouses

Real-time Data Warehouses (DWs) must be able to deal with continuous updates while ensuring 24/7 availability. To improve their performance, distributing data using round-robin algorithms on clusters of shared-nothing machines is normally used. This paper proposes a solution for distributed DW databases that ensures its continuous availability and deals with frequent data loading requirements, while adding small performance overhead. We use a data striping and replication architecture to distribute portions of each fact table among pairs of slave nodes, where each slave node is an exact replica of its partner. This allows balancing query execution and replacing any defective node, ensuring the systems continuous availability. The size of each portion in a given node depends on its individual features, namely performance benchmark measures and dedicated database RAM. The estimated cost for executing each query workload in each slave node is also used for balancing query performance. We include experiments using the TPC-H decision support benchmark to evaluate the scalability of the proposed solution and show that it outperforms standard round-robin distributed DW setups.

24/7 Real-Time Data Warehousing: A Tool for Continuous Actionable Knowledge

Technological evolution has redefined many business models. Many decision makers are now required to act near real-time, instead of periodically, given the latest transactional information. Decision-making occurs much more frequently and considers the latest business data. Since data warehouses (DWs) are the core of business intelligence, decision support systems need to deal with 24/7 real-time requirements. Thus, the ability to deal with continuous data loading and decision support availability simultaneously is critical, for producing continuous actionable knowledge. The main challenge in this context is to efficiently manage the DWs refreshment, when data sources change, to recapture consistency and accuracy with those sources, while maintaining OLAP availability and database performance. This paper proposes a simple, fast and efficient solution based on database replication and temporary tables to change a traditional enterprise DW into a real-time DW, enabling continuous data loading and OLAP availability on a 24/7 schedule. Experimental evaluations using a real-world DW and the TPC-H decision support benchmark show its advantages and analyze its impact in OLAP performance.