Lingyu Xu

Data Prefetching for Scientific Workflow Based on Hadoop

Data-intensive scientific workflow based on Hadoop needs huge data transfer and storage. Aiming at this problem, on the environment of an executing computer cluster which has limited computing resources, this paper adopts the way of data prefetching to hide the overhead caused by data search and transfer and reduce the delays of data access. Prefetching algorithm for data-intensive scientific workflow based on the consideration of available computing resources is proposed. Experimental results indicate that the algorithm consumes less response time and raises the efficiency.

Replica Pre-adjustment Strategy Based on Trend Analysis of File Popularity within Cloud Environment

Data-intensive application within cloud environment needs huge data transfer and migration. In order to optimize the use of data resource, improve the reliability of data and reduce data-access latency and bandwidth consumption, a replica pre-adjustment strategy based on trend analysis of file popularity is proposed. This replica strategy adaptively and dynamically adjust the number of replicas of a file by analyzing the popularity information of the file and comprehensive considering the load balancing and utilization ratio of storage space, which makes the number and distribution of replicas more reasonable, and minimize the costs and maximize the profit of a replica. Experimental results indicate that the algorithm dramatically shortened the execution time of a job and avoided excessive consumption of storage space and decrease overall bandwidth utilization.

Multi-satellite Monitoring SST Data Fusion based on the Adaptive Threshold Clustering Algorithm

This paper proposes a method which describes the information precision with a soft fusion model, instead of the traditional rigid fusion method. The method is divided into two steps, the pretreatment model and fusion center model. Each forms a relative independent model, and the two models have a progressive relationship. The former is used for consistency evaluation, data cleaning and invalid data eliminating, while the latter provides fusion results and variable precision fusion expression by the adaptive threshold clustering algorithm. Experimental results show that the fusion method can not only give every SST data a different precision, but also carry more information to describe precision multiple distribution, which make users get high-quality data and enjoy more rights.

Research and Application of the Intelligent Flow

This paper designs an Intelligent flow which is computed on the cloud platform, the main points are the designation of the Intelligent flow, analyzing and recomposing the Intelligent flow model, and implementation of the program on cloud platform. The experiment is focused on the model execution, dynamic access and fault-tolerant of the intelligent flow executed on the platform. It shows that intelligent flow has a good result test on the cloud platform.

The Research of MEICSP Model through a REST Web Service

Nowadays, Marine Information Systems play a critical role between users and huge marine environmental information. However, it is not easy to sharing of marine environmental information and integration of marine business systems. To integrate systems and share information, it is of great importance to build an easy-accessible platform in parallel with the continuous development of technology innovation and the growth of business requirements of marine industries. In this paper, Combined with the Software as a Service (SaaS) and cloud computing technology, we propose a method for designing a marine environmental information cloud computing services support platform (MEICSP) according to representational state transfer architectural style (REST), and demonstrate the validity of MEICSP through experiments. The contribution of this article is to reveal key aspects of new software architecture for the next generation marine software.

The Research of Building Multidimensional Multi-granularity Automatic Uncertain Knowledge System Based on Attributes Similarity

Network resources are fully rich, but the source of information is uneven. Huge amount of information, complex diversity of the network information and vastly different perspectives has brought great distress for people to identify information [1]. Due to the complexity of objective things, uncertainty and ambiguity of human thinking and other factors, the actual decision-making information is often difficult to quantify. General choice is to express the decision-making information in the form of qualitative language, but this multi-language form depends on human mind. Whats more, different decision-makers will be based on their existing personal experience or cognition degree on the same issue to make good and bad, individualized decision-making, thereby it increases the uncertainty in decision making and labor costs in decision-making process. In this paper, considering the entirety of information on the whole and the drawback of the information on the local, we combine the human knowledge with machine algorithm. The method proposed in this paper is based on the similarity degree of the research objects attributes and categories, which uses the ideas of information fusion [2], make good use of a variety of information together and in view of multidimensional multi-granularity information to confirm each other to find a more effective method to distinguish the similarity measure of information object.

Research on measuring semantic correlation based on the Wikipedia hyperlink network

As a free online encyclopedia with a large-scale of knowledge coverage, rich semantic information and quick update speed, Wikipedia brings new ideas to measure semantic correlation. In this paper, we present a new method for measuring the semantic correlation between words by mining rich semantic information that exists in Wikipedia. Unlike the previous methods that calculate semantic relatedness merely based on the page network or the category network, our method not only takes into account the semantic information of the page network, also combines the semantic information of the category network, and it improve the accuracy of the results. Besides, we analyze and evaluate the algorithm by comparing the calculation results with famous knowledge base (e.g., Hownet) and traditional methods based on Wikipedia on the same test set, and prove its superiority.

A Method of Extracting High Precision Point to Replace in-situ SST Data

Using thirteen satellites sea surface temperature (SST) from infrared radiometers (AVHRR, MODIS) and microwave radiometers (TMI, AMSR), a set of high-quality, cloud-free, high-resolution(0.25°) merged SST product was generated through a method of extended revisal point. The area of merged SST product and extended revisal point product [15.875-40.125°N, 105.875-130.125°E] cover the whole China Sea. The experimental results show that the number of extension correct point will be hundreds of thousands times as many as in-situ SST data. It only has a small deviation by comparison with in-situ SST data, and the precision between the two is similar. Therefore, extension correct point can replace in-situ data under certain conditions. This study will offer a wider SST reference corrections data for quality evaluation, multi-source remote sensing, and assimilation technology.

Data De-duplication for Primary Storage System

Data De-duplication has being used comprehensively within Disk-based Backups, Archives and Disaster Recovery. This technologys high data reduction ratio and satisfied performance in secondary data storage system has aroused huge interests in other fields. The primary storage system is indeed a field being long for data reduction. Data de-duplication maybe a chance to alleviate data redundancy problem within that field. Actually, there are still lots of problems and technology challenges in resolving the contradiction between strict performance requirements in the primary storage system and the resources consuming character of the data de-duplication. This article will give an introduction about those technology challenges and propose a new data storage framework trying to improve those problems.