Viktor Medvedev

Strategies for Big Data Clustering

In the paper, an overview of methods and technologies used for big data clustering is presented. The clustering is one of the important data mining issue especially for big data analysis, where large volume data should be grouped. Here some clustering methods are described, great attention is paid to the k-means method and its modifications, because it still remains one of the popular methods and is implemented in innovative technologies for big data analysis. Neural network-based self-organizing maps and their extensions for big data clustering are reviewed, too. Some strategies for big data clustering are also presented and discussed. It is shown the data of which volume can be clustered in the well known data mining systems WEKA and KNIME and when new sophisticated technologies are needed.

Optimization of the Local Search in the Training for SAMANN Neural Network

In this paper, we discuss the visualization of multidimensional data. A well-known procedure for mapping data from a high-dimensional space onto a lower-dimensional one is Sammon’s mapping. This algorithm preserves as well as possible all interpattern distances. We investigate an unsupervised backpropagation algorithm to train a multilayer feed-forward neural network (SAMANN) to perform the Sammon’s nonlinear projection. Sammon mapping has a disadvantage. It lacks generalization, which means that new points cannot be added to the obtained map without recalculating it. The SAMANN network offers the generalization ability of projecting new data, which is not present in the original Sammon’s projection algorithm. To save computation time without losing the mapping quality, we need to select optimal values of control parameters. In our research the emphasis is put on the optimization of the learning rate. The experiments are carried out both on artificial and real data. Two cases have been analyzed: (1) training of the SAMANN network with full data set, (2) retraining of the network when the new data points appear.

Large Datasets Visualization with Neural Network Using Clustered Training Data

This paper presents the visualization of large datasets with SAMANN algorithm using clustering methods for initial dataset reduction for the network training. The visualization of multidimensional data is highly important in data mining because recent applications produce large amount of data that need specific means for the knowledge discovery. One of the ways to visualize multidimensional dataset is to project it onto a plane. This paper analyzes the visualization of multidimensional data using feed-forward neural network. We investigate an unsupervised backpropagation algorithm to train a multilayer feed-forward neural network (SAMANN) to perform the Sammon`s nonlinear projection. The SAMANN network offers the generalization ability of projecting new data. Previous investigations showed that it is possible to train SAMANN using only a part of analyzed dataset without the loss of accuracy. It is very important to select proper vector subset for the neural network training. One of the ways to construct relevant training subset is to use clustering. This allows to speed up the visualization of large datasets.

Parallel Realizations of the SAMANN Algorithm

Sammons mapping is a well-known procedure for mapping data from a higher-dimensional space onto a lower-dimensional one. But the original algorithm has a disadvantage. It lacks generalization, which means that new points cannot be added to the obtained map without recalculating it. The SAMANN neural network, that realizes Sammons algorithm, provides a generalization capability of projecting new data. A drawback of using SAMANN is that the training process is extremely slow. One of the ways of speeding up the neural network training process is to use parallel computing. In this paper, we proposed some parallel realizations of the SAMANN.

The Problem of Visual Analysis of Multidimensional Medical Data

We consider the problem of visual analysis of the multidimensional medical data. A frequent problem in medicine is an assignment of a health state to one of the known classes (for example, healthy or sick persons). A particularity of medical data classification is the fact that the transit from the normal state to diseased one is often not so conspicuous. From the table of the parametric medical multidimensional data, it is difficult to notice which objects are similar, which ones are different, i.e., which class they belong to. Therefore it is necessary to classify the multidimensional data by various classification methods. However, classification errors arc inevitable and the results of classification in medicine must be as correct as possible. That is why it is advisable to use different types of data analysis methods, for example, in addition to visualize the multidimensional data (to project to a plane). A visual analysis allows us to estimate similarities and differences of objects, a partial assignment to one or another class in simple visual way. However, the shortcoming of this analysis is the fact that while projecting multidimensional data to a plane, a part of the information is inevitably lost. Thus, one of the agreeable methods is a combination of classification and visualization methods. This synthesis lets us to obtain a more objective conclusions on the analysed data. The results, obtained by the integrated method, proposed in this chapter, can help medics to preliminary diagnose successfully or have some doubt on the former diagnosis.

Large-scale multidimensional data visualization: a web service for data mining

In this paper, we present an approach of the Web application (as a service) for data mining oriented to the multidimensional data visualization. The stress is put on visualization methods as a tool for the visual presentation of large-scale multidimensional data sets. The proposed implementation includes five visualization methods: MDS SMACOF algorithm, Relative MDS, Diagonal majorization algorithm, Relational perspective map, SAMANN. A cluster for parallel computation is used by Web service for the visual data mining. The service is of free access to the user community for data visualization.

Web Application for Large-Scale Multidimensional Data Visualization

Abstract In this paper, we present an approach of the web application (as a service) for data mining oriented to the multidimensional data visualization. This paper focuses on visualization methods as a tool for the visual presentation of large-scale multidimensional data sets. The proposed implementation of such a web application obtains a multidimensional data set and as a result produces a visualization of this data set. It also supports different configuration parameters of the data mining methods used. Parallel computation has been used in the proposed implementation to run the algorithms simultaneously on different computers.

A new web-based solution for modelling data mining processes

Abstract The conventional technologies and methods are not able to store and analyse recent data that come from different sources: various devices, sensors, networks, transactional applications, the web, and social media. Due to a complexity of data, data mining methods should be implemented using the capabilities of the Cloud technologies. In this paper, a new web-based solution named DAMIS, inspired by the Cloud, is proposed and implemented. It allows making massive data mining simpler, effective, and easily understandable for data scientists and business intelligence professionals by constructing scientific workflows for data mining using a drag and drop interface. The usage of scientific workflows allows composing convenient tools for modelling data mining processes and for simulation of real-world time- and resource-consuming data mining problems. The solution is useful to solve data classification, clustering, and dimensionality reduction problems. The DAMIS architecture is designed to ensure easy accessibility, usability, scalability, and portability of the solution. The proposed solution has a wide range of applications and allows to get deep insights into the data during the process of knowledge discovery.

Cloud Technologies: A New Level for Big Data Mining

Nowadays, the amount of data being collected and stored has been constantly increasing. Data come from different sources such as various devices, sensors, networks, transactional applications, web and social media. Conventional technologies and methods are not able to store and analyze such amount of data. In this paper, a comparative analysis of the existing data mining systems is performed and it shows that the most of existing data mining solutions are not appropriate to solve Big Data problems. In order to bring conventional data mining to a new level and to cope with challenges of massive and complex data of different nature, requirements for data mining systems suitable for Big Data are derived.

Cloud Computing Approach for Intelligent Visualization of Multidimensional Data

In this paper, a Cloud computing approach for intelligent visualization of multidimensional data is proposed. Intelligent visualization enables to create visualization models based on the best practices and experience. A new Cloud computing-based data mining system DAMIS is introduced for the intelligent data analysis including data visualization methods. It can assist researchers to handle large amounts of multidimensional data when executing resource-expensive and time-consuming data mining tasks by considerably reducing the information load. The application of DAMIS is illustrated by the visual analysis of medical streaming data.