Noel Novelli

FUN: An Efficient Algorithm for Mining Functional and Embedded Dependencies

Discovering functional dependencies from existing databases is an important technique strongly required in database design and administration tools. Investigated for long years, such an issue has been recently addressed with a data mining viewpoint, in a novel and more efficient way by following from principles of level-wise algorithms. In this paper, we propose a new characterization of minimal functional dependencies which provides a formal framework simpler than previous proposals. The algorithm, defined for enforcing our approach has been implemented and experimented. It is more efficient (in whatever configuration of original data) than the best operational solution (according to our knowledge): the algorithm Tane. Moreover, our approach also performs (without additional execution time) the mining of embedded functional dependencies, i.e. dependencies holding for a subset of the attribute set initially considered (e.g. for materialized views widely used in particular for managing data warehouses).

Functional and embedded dependency inference: a data mining point of view

Abstract The issue of discovering functional dependencies from populated databases has received a great deal of attention because it is a key concern in database analysis. Such a capability is strongly required in database administration and design while being of great interest in other application fields such as query folding. Investigated for long years, the issue has been recently addressed in a novel and more efficient way by applying principles of data mining algorithms. The two algorithms fitting in such a trend are T ANE and Dep-Miner. They strongly improve previous proposals. In this paper, we propose a new approach adopting a data mining point of view. We define a novel characterization of minimal functional dependencies. This formal framework is sound and simpler than related work. We introduce the new concept of free set for capturing source of functional dependencies. By using the concepts of closure and quasi-closure of attribute sets, targets of such dependencies are characterized. Our approach is enforced through the algorithm F UN which is particularly efficient since it is comparable or improves the two best operational solutions (according to our knowledge): T ANE and Dep-Miner. It makes use of various optimization techniques and it can work on very large databases. Applying on real life or synthetic data more or less correlated, comparative experiments are performed in order to assess performance of F UN against T ANE and Dep-Miner. Moreover, our approach also exhibits (without significant additional execution time) embedded functional dependencies, i.e. dependencies captured in any subset of the attribute set originally considered. Embedded dependencies capture a knowledge specially relevant in all fields where materialized data sets are managed (e.g. materialized views widely used in data warehouses).

Visually Mining the Datacube using a Pixel-Oriented Technique

This paper introduces a new technique easing the navigation and interactive exploration of huge multidimensional datasets. Following the pixel-oriented paradigm [8], the key ingredients enabling the interactive navigation of extreme volumes of data rely on a set of functions bijectively mapping data elements to screen pixels. The use of the mapping from data elements to pixels constrain the computational complexity for the rendering process to be linear with respect to the number of rendered pixels on the screen as opposed to the dataset size. Our method furthermore allows the implementation of usual information visualization techniques such as zoom and pan, anamorphosis and texturing. As a proof-of-concept, we show how our technique can be adapted to interactively explore the Datacube, turning our approach into an efficient system for visual datamining. We report experiments conducted on a Datacube containing 50 millions of items. To our knowledge, our technique outperforms all existing ones and push the scalability limit close to the billion of elements. Supporting all basic navigation techniques, and being moreover flexible makes it easily reusable for a large number of applications.

Summarizing Datacubes: Semantic and Syntactic Approaches

Datacubes are especially useful for answering efficiently queries on data warehouses. Nevertheless the amount of generated aggregated data is huge with respect to the initial data which is itself very large. Recent research work has addressed the issue of summarizing Datacubes in order to reduce their size. In this chapter, we present three different approaches. They propose structures which make it possible to reduce the size of the data cube representation. The two former, the closed cube and the quotient cube, are said semantic and discard the redundancies captured within data cubes. The size of the underlying representations is especially reduced but the counterpart is an additional response time when answering the OLAP queries. The latter approach is rather syntactic since it enforces an optimization at the logical level. It is called Partition Cube and based on the concept of partition. We also give an algorithm to compute it. We propose a Relational Partition Cube, a novel R-Olap cubing solution for managing Partition Cubes using the relational technology. An analytical evaluation shows that the storage space of Partition Cubes is smaller than Datacubes. In order to confirm analytical comparison, experiments are performed in order to compare our approach with Datacubes and with two of the best reduction methods, the Quotient Cube and the Closed Cube.

Calcul et fouille visuelle orientée-pixel de cubes de données

DataCubes (DC) are especially useful for answering efficiently queries on data warehouses. On the one hand, several approaches offer different solutions to compute DC and another research have focused on data visualization to exploit the data in an interactive ways. We proposed a condensed representation of DC allowing to effectively compute the DC and to represent the relations between data for the visualization. The visualization proposed in the paper is based on the pixel oriented visualization techniques and a link node diagram technique that enables both global and local exploration. The approach is a new approach that uses the effective compute of the datacubes on the one hand and the advanced techniques of visualization on the other hand.

Lossless reduction of datacubes

Datacubes are specially useful for answering efficiently queries on data warehouses. Nevertheless the amount of generated aggregated data is incomparably more voluminous than the initial data which is itself very large. Recently, research work has addressed the issue of a concise representation of datacubes in order to reduce their size. The approach presented in this paper fits in a similar trend. We propose a concise representation, called Partition Cube, based on the concept of partition and define an algorithm to compute it. Various experiments are performed in order to compare our approach with methods fitting in the same trend. This comparison relates to the efficiency of algorithms computing the representations, the main memory requirements, and the storage space which is necessary.