Corinne Grac

Multidimensional modeling and analysis of large and complex watercourse data: an OLAP-based solution

Abstract This paper presents the application of Data Warehouse (DW) and On-Line Analytical Processing (OLAP) technologies to the field of water quality assessment. The European Water Framework Directive (DCE, 2000) underlined the necessity of having operational tools to help in the interpretation of the complex and abundant information regarding running waters and their functioning. Several studies have exemplified the interest in DWs for integrating large volumes of data and in OLAP tools for data exploration and analysis. Based on free software tools, we propose an extensible relational OLAP system for the analysis of physicochemical and hydrobiological watercourse data. This system includes: (i) two data cubes; (ii) an Extract, Transform and Load (ETL) tool for data integration; and (iii) tools for OLAP exploration. Many examples of OLAP analysis (thematic, temporal, spatiotemporal, and multiscale) are provided. We have extended an existing framework with complex aggregate functions that are used to define complex analysis indicators. Additional analysis dimensions are also introduced to allow their calculation and also for purposes of rendering information. Finally, we propose two strategies to address the problem of summarizing heterogeneous measurement units by: (i) transforming source data at the ETL tier, and (ii) introducing an additional analysis dimension at the OLAP server tier.

Discriminant temporal patterns for linking physico-chemistry and biology in hydro-ecosystem assessment

We propose a new data mining process to extract original knowledge from hydro-ecological data, in order to help the identification of pollution sources. This approach is based (1) on a domain knowledge discretization (quality classes) of physico-chemical and biological parameters, and (2) on an extraction of temporal patterns used as discriminant features to link physico-chemistry with biology in river sampling sites. For each bio-index quality value, we obtained a set of significant discriminant features. We used them to identify the physico-chemical characteristics that impact on different biological dimensions according to their presence in extracted knowledge. The experiments meet with the domain knowledge and also highlight significant mismatches between physico-chemical and biological quality classes. Then, we discuss about the interest of using discriminant temporal patterns for the exploration and the analysis of temporal environmental data such as hydro-ecological databases.

Performance-friendly rule extraction in large water data-sets with AOC posets and relational concept analysis

In this paper, we consider data analysis methods for knowledge extraction from large water data-sets. More specifically, we try to connect physico-chemical parameters and the characteristics of taxons living in sample sites. Among these data analysis methods, we consider formal concept analysis (FCA), which is a recognized tool for classification and rule discovery on object–attribute data. Relational concept analysis (RCA) relies on FCA and deals with sets of object–attribute data provided with relations. RCA produces more informative results but at the expense of an increase in complexity. Besides, in numerous applications of FCA, the partially ordered set of concepts introducing attributes or objects (AOC poset, for Attribute–Object–Concept poset) is used rather than the concept lattice in order to reduce combinatorial problems. AOC posets are much smaller and easier to compute than concept lattices and still contain the information needed to rebuild the initial data. This paper introduces a variant of the RCA process based on AOC posets rather than concept lattices. This approach is compared with RCA based on iceberg lattices. Experiments are performed with various scaling operators, and a specific operator is introduced to deal with noisy data. We show that using AOC poset on water data-sets provides a reasonable concept number and allows us to extract meaningful implication rules (association rules whose confidence is 1), whose semantics depends on the chosen scaling operator.

Experimental study of the uncertainty of the intrasubstrate variability on two French index metrics based on macroinvertebrates

Chemical and biological assessments of waterbodies are required to fulfil the objectives of the European Water Framework Directive (EU Off J 327:1–72, 2000). Many past studies have been focused on water chemistry uncertainties, but few have been carried out on the hydrobiological aspects. Considerable research on macroinvertebrates has highlighted the mosaic habitat impact on the macroinvertebrate distribution, substrate type and within-substrate heterogeneity as uncertainty sources. We thus studied the effect of substrate variability on the metrics of two French biological indices (normalized index IBGN and new index I2M2 metrics) using experimental field sampling based on substrate replicates. For each of the nine substrates studied, a minimum of 2 sites were selected (among a total of 31 sites) in 7 hydroecoregions (HER). Ten replicates per substrate and site were collected where possible. We obtained 315 faunistic lists associated with 315 substrate replicates. Twelve metrics, such as the Shannon index and variety class, were calculated by list. We used multidimensional scaling and similarity indices to analyse the substrate variability effect. Our results highlighted the extent of HER and site features on the within-substrate heterogeneity. Faunistic lists and metrics varied more in root and sand substrates than in other substrates. I2M2 metrics, i.e. ASPT, taxonomic richness or H’, calculated on each replicate, eliminated the intrasubstrate variability.

Principled Data Preprocessing: Application to Biological Aquatic Indicators of Water Pollution

In many biological studies, statistical and data mining methods are extensively used to analyze the data and discover actionable knowledge. But, bad data quality causing incorrect analysis results and wrong interpretations may induce misleading conclusions and inadequate decisions. To ensure the validity of the results, avoid bias and data misuse, it is necessary to control not only the whole analytical pipeline, but most importantly the quality of the data with appropriate data preprocessing choices. Since various preprocessing techniques and alternative strategies may lead to dramatically different outputs, it is crucial to rely on a principled and rigorous method to select the optimal set of data preprocessing steps that depends both on the input data distributional characteristics and on the inherent characteristics of the targeted statistical or data mining methods. In this paper, we propose a method that selects, given a dataset, the optimal set of preprocessing tasks to apply to the data such that the overall data preprocessing output maximizes the quality of the analytical results for various techniques of clustering, regression, and classification. We present some promising results that validate our approach on biomonitoring data preparation.

Un système décisionnel pour l’analyse de la qualité des eaux de rivières

Cet article decrit un systeme decisionnel developpe pour permettre l’analyse des donnees concernant le fonctionnement des hydro-ecosystemes ; ces donnees sont nombreuses, diverses et issues de sources variees. Le systeme mis en place comporte une base de donnees integree, un entrepot permettant l’exploration des dimensions associees aux donnees, et des outils de fouille permettant de repondre aux questions des hydro-ecologues.