Boriana L. Milenova

O-Cluster: scalable clustering of large high dimensional data sets

Clustering large data sets of high dimensionality has always been a challenge for clustering algorithms. Many recently developed clustering algorithms have attempted to address either handling data sets with a very large number of records and/or with a very high number of dimensions. We provide a discussion of the advantages and limitations of existing algorithms when they operate on very large multidimensional data sets. To simultaneously overcome both the curse of dimensionality and the scalability problems associated with large amounts of data, we propose a new clustering algorithm called O-Cluster. O-Cluster combines a novel active sampling technique with an axis-parallel partitioning strategy to identify continuous areas of high density in the input space. The method operates on a limited memory buffer and requires at most a single scan through the data. We demonstrate the high quality of the obtained clustering solutions, their robustness to noise, and O-Clusters excellent scalability.

Data-centric automated data mining

Data mining is a difficult task. It requires complex methodologies, including problem definition, data preparation, model selection, and model evaluation. This has limited the adoption of data mining at large and in the database and business intelligence (BI) communities more specifically. The concepts and methodologies used in data mining are foreign to database and BI users in general. This paper proposes a new approach to the design of data mining applications targeted at these user groups. This approach uses a data-centric focus and automated methodologies to make data mining accessible to nonexperts. The automated methodologies are exposed through high-level interfaces. This hides the data mining concepts away from the users thus helping to bridge the conceptual gap usually associated with data mining. We illustrate the approach with two applications: the new Oracle Predictive Analytics feature of Oracle Database 10g Release 2 and the Oracle Spreadsheet Add-In for Predictive Analytics.

Oracle Data Mining

Oracle has completed a major research and development effort to add native Data Mining and pattern recognition algorithms to the Oracle RDBMS. As a result, Oracle Data Mining (ODM) provides a comprehensive collection of Data Mining analytics as part of the Oracle database environment that supports the development, integration and deployment of Data Mining applications. This Data Mining infrastructure has a native SQL and PL/SQL API but can also be accessed from a Java API or the ODM user interface. ODM enables data analysts and developers to discover insights hidden in their data and create advanced Data Mining applications that extend the benefits of DMa Mining to many users throughout an organization. ODM leverages the powerful and feature-rich Oracle RDBMS environment including comprehensive capabilities for data storage, data preparation and processing, information retrieval, scalability, security, transaction control, parallelism, versioning, workflow, and reliability. In this article, we describe the functionality and algorithms behind ODM and the advantages of the Data Mining in the database paradigm. We conclude with two examples of the use of ODM: a SVM methodology for tumor classification and the integration of Naive Bayes predictive models in Oracle’s marketing business application (Oracle Marketing).

Creation and deployment of data mining-based intrusion detection systems in Oracle Database l0g

Network security technology has become crucial in protecting government and industry computing infrastructure. Modern intrusion detection applications face complex requirements - they need to be reliable, extensible, easy to manage, and have low maintenance cost. In recent years, data mining-based intrusion detection systems (IDSs) have demonstrated high accuracy, good generalization to novel types of intrusion, and robust behavior in a changing environment. Still, significant challenges exist in the design and implementation of production quality IDSs. Instrumenting components such as data transformations, model deployment, and cooperative distributed detection remain a labor intensive and complex engineering endeavor. This paper describes DAID, a database-centric architecture that leverages data mining within the Oracle RDBMS to address these challenges. DAID also offers numerous advantages in terms of scheduling capabilities, alert infrastructure, data analysis tools, security, scalability, and reliability. DAID is illustrated with an Intrusion Detection Center application prototype that leverages existing functionality in Oracle Database 10g.

Mining high-dimensional data for information fusion: a database-centric approach

Data mining on high-dimensional heterogeneous data is a crucial component in information fusion application domains such as remote sensing, surveillance, and homeland security. The information processing requirements of these domains place a premium on security, robustness, performance, and sophisticated analytic methods. This paper introduces a database-centric approach that enables data mining and analysis of data that typically interest the information fusion community. The approach benefits from the inherent security, reliability, and scalability found in contemporary RDBMSs. The capabilities of this approach are demonstrated on satellite imagery. Hyperspectral data are mined using clustering (O-Cluster) and classification (Support Vector Machines) techniques. The data mining is performed inside the database, which ensures maintenance of data integrity and security throughout the analytic effort. Within the database, the clustering and classification results can be further combined with spatial processing components to enable additional analysis.