Madina Mansurova

Distributed parallel algorithm for numerical solving of 3D problem of fluid dynamics in anisotropic elastic porous medium using MapReduce and MPI technologies

Paper presents an advanced iterative MapReduce solution that employs Hadoop and MPI technologies. First, we present an overview of working implementations that make use of the same technologies. Then we define an academic example of numeric problem with an emphasis on its computational features. The named definition is used to justify the proposed solution design.

Automatic Generation of Concept Maps based on Collection of Teaching Materials

The aim of this work is demonstration of usefulness and efficiency of statistical methods of text processing for automatic construction of concept maps of the pre-determined domain. Statistical methods considered in this paper are based on the analysis of co-occurrence of terms in the domain documents. To perform such analysis, at the first step we construct a term-document frequency matrix on the basis of which we can estimate the correlation between terms and the designed domain. At the second step we go on from the term-document matrix to the term-term matrix that allows to estimate the correlation between pairs of terms. The use of such approach allows to define the links between concepts as links in pairs which have the highest values of correlation. At the third step, we have to summarize the obtained information identifying concepts as nodes and links as edges of a graph and construct a concept map as resulting graph.

Design and Development of Media-Corpus of the Kazakh Language

The aim of this work was design and development of a media-corpus of the Kazakh language. The media-corpus is hosted by the al-Farabi Kazakh National University and serves linguists as an empirical basis for research on contemporary written Kazakh. The information system for media-corpus was built on the basis of component software architecture. To make the processes of collection, storage and analysis of media-texts in the Kazakh language automatic, four components of the information system were designed and developed. The text files are saved in XML format. At the stage of analysis such tasks as text normalization, removing stop words, adding metadata and morphological analysis are performed. The morphological analyzer receives an input of a plain text, and at the output gives the text in XML format, which is further convenient to work with as it is easily converted to JSON format. The XML format is defined using XML Schema Definition (XSD). XSD allows to convert data into any other format, which simplifies the data exchange between the systems. For the case of incomplete morphological markup and the presence of homonymy, a special interface to perform manual markup is developed.

An Approach of Automatic Extraction of Domain Keywords from the Kazakh Text

In this paper we consider the approach of automatic extraction of domain keywords from the Kazakh Text based on statistical methods of natural language processing. The proposed approach can be used to build domain dictionaries and thesauri without manual work of domain experts. Results of experiments on a corpus of texts from a Kazakh book and online websites demonstrate that applying latent semantic analysis to keywords extraction significantly decreases information noise and strengthens the words relations.

A New Operationalization of Contrastive Term Extraction Approach Based on Recognition of Both Representative and Specific Terms

A contrastive approach to term extraction is an extensive class of methods based on the assumption that the words frequently occurring within a domain and rarely beyond it are most likely terms. The disadvantage of this approach is a great number of type II errors – false negatives. The cause of these errors is in the idea of contrastive selection when the most representative high frequent terms are extracted from the texts and rare terms are discarded. In this work, we propose a new operationalization of the contrastive approach, which supports the capture of both high frequent and low frequent domain terms. Proposed operationalization reduces the number of false negatives. The experiments performed on the texts of the subject domain “Geology” show promising of proposed approach.

Development of a Distributed Parallel Algorithm of 3D Hydrodynamic Calculation of Oil Production on the Basis of MapReduce Hadoop and MPI Technologies

The developed hybrid model of high performance computing and the realized applications on the basis of MapReduce Hadoop and MPI technologies allow to solve effectively the different classes of oil production problems. Investigations of high performance computing to solve the problem of 3D hydrodynamic calculation of oil production resulted in proposition of a constructive approach to organization of distributed parallel computing using MapReduce Hadoop and MPI technologies for which a general scheme of the iteration infrastructure of MapReduce model is designed; the structure for fulfillment of map and reduce methods and organization of decomposition of the computational area at different Map/Reduce stages are presented; a computational experiment on specific infrastructure is carried out. As the result of this work the architecture of the system realized on the advantages of MapReduce Hadoop and MPI technologies is constructed.

Iterative MapReduce MPI oil reservoir simulator

Paper presents an advanced Iterative MapReduce MPI oil reservoir simulator. First we present an overview of working implementations that make use of the same technologies. Then we define an academic example of numeric problem with an emphasis on its computational features. We present a distributed parallel algorithm of hybrid solution of the problem using MapReduce Hadoop and MPI technologies and describe an improved variant of the algorithm using memory-mapped files.

An Automatic Construction of Concept Maps Based on Statistical Text Mining

In this paper, we explore the task of automatic construction of concept maps for various knowledge domains. We propose a simple 3-steps algorithm for extraction of key elements of a concept map (nodes and links) from a given collection of domain documents. Our algorithm manipulates a statistical term-document matrix describing how frequently terms occur in documents of the collection. At the first step we decompose this matrix into scores (terms-by-factors) and loadings (factors-by-documents) matrixes using non-negative matrix factorization, wherein each factor represents one topic of the collection. Since the scores matrix specifies the relative contribution of each term to the factors, we can select the most contributing terms and use them as concept map nodes. At the second step we associate selected key terms with the corresponding row-vectors of the term-document matrix and calculate pairwise cosine distances between them. Since the close distances determine the pairs of strongly related key terms, we can select the strongest relations as concept map links. Finally, we construct the resulting concept map as a graph with selected nodes and links. The benefits of our statistical algorithm are its simplicity, efficiency and applicability to any domain, any language and any document collection.

Applying MDA to Generate Hadoop Based Scientific Computing Applications

The paper presents an attempt to develop and deploy a functioning MDA (Model-Driven Architecture) model of a distributed scientific application. The main focus is a problem of modeling high performance computing processes in a visual notation and automatic generation of an executable code using the resulting diagrams. The article describes the efforts to create a platform independent model of process execution, transformation it into a platform specific model and, finally, automatic generation an application code. The research novelty includes a platform independent model of the classic hydrodynamics problem, equivalent Hadoop based platform specific model and the testing results that confirm feasibility of the research.