Zhipeng Gui

Evaluating open-source cloud computing solutions for geosciences

Many organizations start to adopt cloud computing for better utilizing computing resources by taking advantage of its scalability, cost reduction, and easy to access characteristics. Many private or community cloud computing platforms are being built using open-source cloud solutions. However, little has been done to systematically compare and evaluate the features and performance of open-source solutions in supporting Geosciences. This paper provides a comprehensive study of three open-source cloud solutions, including OpenNebula, Eucalyptus, and CloudStack. We compared a variety of features, capabilities, technologies and performances including: (1) general features and supported services for cloud resource creation and management, (2) advanced capabilities for networking and security, and (3) the performance of the cloud solutions in provisioning and operating the cloud resources as well as the performance of virtual machines initiated and managed by the cloud solutions in supporting selected geoscience applications. Our study found that: (1) no significant performance differences in central processing unit (CPU), memory and I/O of virtual machines created and managed by different solutions, (2) OpenNebula has the fastest internal network while both Eucalyptus and CloudStack have better virtual machine isolation and security strategies, (3) Cloudstack has the fastest operations in handling virtual machines, images, snapshots, volumes and networking, followed by OpenNebula, and (4) the selected cloud computing solutions are capable for supporting concurrent intensive web applications, computing intensive applications, and small-scale model simulations without intensive data communication.

A Service Brokering and Recommendation Mechanism for Better Selecting Cloud Services

Cloud computing is becoming the new generation computing infrastructure, and many cloud vendors provide different types of cloud services. How to choose the best cloud services for specific applications is very challenging. Addressing this challenge requires balancing multiple factors, such as business demands, technologies, policies and preferences in addition to the computing requirements. This paper recommends a mechanism for selecting the best public cloud service at the levels of Infrastructure as a Service (IaaS) and Platform as a Service (PaaS). A systematic framework and associated workflow include cloud service filtration, solution generation, evaluation, and selection of public cloud services. Specifically, we propose the following: a hierarchical information model for integrating heterogeneous cloud information from different providers and a corresponding cloud information collecting mechanism; a cloud service classification model for categorizing and filtering cloud services and an application requirement schema for providing rules for creating application-specific configuration solutions; and a preference-aware solution evaluation mode for evaluating and recommending solutions according to the preferences of application providers. To test the proposed framework and methodologies, a cloud service advisory tool prototype was developed after which relevant experiments were conducted. The results show that the proposed system collects/updates/records the cloud information from multiple mainstream public cloud services in real-time, generates feasible cloud configuration solutions according to user specifications and acceptable cost predication, assesses solutions from multiple aspects (e.g., computing capability, potential cost and Service Level Agreement, SLA) and offers rational recommendations based on user preferences and practical cloud provisioning; and visually presents and compares solutions through an interactive web Graphical User Interface (GUI).

A performance, semantic and service quality-enhanced distributed search engine for improving geospatial resource discovery

Geospatial resource discovery is a critical step for developing geographic science applications. With the increasing number of geospatial resources available online, many Spatial Data Infrastructure (SDI) components (e.g. catalogues and portals) have been developed to help manage and discover geospatial resources. However, efficient and accurate geospatial resource discovery is still a big challenge because of the heterogeneity and complexity of decentralized network environments and interdisciplinary semantics. In this article, we report a search engine framework for efficient geospatial resource discovery, which reduces integration costs by leveraging existing Geospatial Cyberinfrastructure (GCI) components. Specifically, (1) the framework provides integration capability and flexibility by adopting the brokering approach, implementing a ‘plug-in’-based framework for metadata processing and proposing a dynamically configurable search workflow; (2) the asynchronous messaging and batch processing-based metadata record retrieval mode enhances the search performance and user interactivity; (3) an embedded semantic support system improves the discovery recall level and precision by providing semantic-based search rule creation and result similarity evaluation functions and (4) the engine assists user decision-making by integrating a service quality monitoring and evaluation system, data/service visualization tools, multiple views and additional information. Experiments and a search example show that the proposed engine helps both scientists and general users search for more accurate results with enhanced performance and user experience through a user-friendly interface.

Adopting cloud computing to optimize spatial web portals for better performance to support Digital Earth and other global geospatial initiatives

A spatial web portal (SWP) provides a web-based gateway to discover, access, manage, and integrate worldwide geospatial resources through the Internet and has the access characteristics of regional to global interest and spiking. Although various technologies have been adopted to improve SWP performance, enabling high-speed resource access for global users to better support Digital Earth remains challenging because of the computing and communication intensities in the SWP operation and the dynamic distribution of end users. This paper proposes a cloud-enabled framework for high-speed SWP access by leveraging elastic resource pooling, dynamic workload balancing, and global deployment. Experimental results demonstrate that the new SWP framework outperforms the traditional computing infrastructure and better supports users of a global system such as Digital Earth. Reported methodologies and framework can be adopted to support operational geospatial systems, such as monitoring national geographic state and spanning across regional and global geographic extent.

A Data Dependency Relationship Directed Graph and Block Structures Based Abstract Geospatial Information Service Chain Model

With the development of Web services and service-oriented architecture (SOA), Web services has become the primary method to implement geospatial information sharing and interoperability, besides, geospatial information processing modeling basing on Web service compositions is one of the research hotspots. The data-centered procedure of geospatial information processing describes the steps of data processing and information retrieval. Current Web service composition languages and workflow models no matter low-level IT-oriented or high-level abstract are all with complex models, can not express data and data flow intuitively and not suit for geospatial domain users. In this paper, a data-dependency directed graph and block structures based abstract geospatial information service chain model (DDBASCM) is proposed and the translation method to BPEL is also suggested, geospatial domain users who are not Web services experts can modeling service chains intuitively, translate service chain into BPEL executable process and execute it using BPEL engine.

Optimizing an index with spatiotemporal patterns to support GEOSS Clearinghouse

A variety of Earth observation systems monitor the Earth and provide petabytes of geospatial data to decision-makers and scientists on a daily basis. However, few studies utilize spatiotemporal patterns to optimize the management of the Big Data. This article reports a new indexing mechanism with spatiotemporal patterns integrated to support Big Earth Observation (EO) metadata indexing for global user access. Specifically, the predefined multiple indices mechanism (PMIM) categorizes heterogeneous user queries based on spatiotemporal patterns, and multiple indices are predefined for various user categories. A new indexing structure, the Access Possibility R-tree (APR-tree), is proposed to build an R-tree-based index using spatiotemporal query patterns. The proposed indexing mechanism was compared with the classic R*-tree index in a number of scenarios. The experimental result shows that the proposed indexing mechanism generally outperforms a regular R*-tree and supports better operation of Global Earth Observation System of Systems (GEOSS) Clearinghouse.

Geospatial Service Web: towards integrated cyberinfrastructure for GIScience

A geospatial cyberinfrastructure is needed to support advanced GIScience research and education activities. However, the heterogeneous and distributed nature of geospatial resources creates enormous obstacles for building a unified and interoperable geospatial cyberinfrastructure. In this paper, we propose the Geospatial Service Web (GSW) to underpin the development of a future geospatial cyberinfrastructure. The GSW excels over the traditional spatial data infrastructure by providing a highly intelligent geospatial middleware to integrate various geospatial resources through the Internet based on interoperable Web service technologies. The development of the GSW focuses on the establishment of a platform where data, information, and knowledge can be shared and exchanged in an interoperable manner. Theoretically, we describe the conceptual framework and research challenges for GSW, and then introduce our recent research toward building a GSW. A research agenda for building a GSW is also presented in the paper.

Contemporary Computing Technologies for Processing Big Spatiotemporal Data

Geographic phenomena evolve in a four-dimensional spatiotemporal world. To capture the geographical phenomena at different scales, large amount of data (big data) are produced with specific spatiotemporal patterns. Phenomena evolution and the principles driving the evolution provide pathways for developing methodological solutions to process the big spatiotemporal data. Based on experiences gained from several projects, such as climate studies and cloud computing, we introduce in this chapter modern computing technologies required for processing big data, including (1) sensor web, Earth observations, and model simulations for collecting and generating big data, (2) flexible and standard-based systems for managing big data for easy discovery and access, (3) multidimensional visual analytics for exploring and analyzing big spatiotemporal data, and (4) grid, cloud, and GPU computing for addressing the computing intensive challenges. We discuss through exemplar projects how these cutting-edge computing technologies are utilized to handle big spatiotemporal data. We expect this chapter to set a computing research context for future big data handling at different spatiotemporal granules.

An experimental study of open-source cloud platforms for dust storm forecasting

Cloud computing is becoming a viable computing solution for scientific research and several open-source cloud solutions are available to support scientific studies. However, little has been done to systematically investigate the performance of these solutions in supporting scientific pursuits. Taking dust storm forecasting as an example, we test three popular open-source cloud solutions, namely Eucalyptus, OpenNebula, and CloudStack, on the same hardware and compare against a bare cluster. We find that: (1) compared to the bare cluster, a cloud has about 10% virtualization and management overhead when one virtual machine is used. Overhead increases when more virtual machines are used. Leveraging more virtual resources would not necessarily yield better performance. (2) For computing- and communication-intensive dust storm forecasting, the performance overhead is mainly due to virtualized network rather than virtualized computing resources when more than one virtual machine is involved. (3) Compared to Eucalyptus and CloudStack, OpenNebula provides better support for dust storm forecasting with relatively better performance. The results can provide some insights for scientific community in adopting these open-source cloud solutions.

GeoSquare: collaborative geoprocessing models’ building, execution and sharing on Azure Cloud

Collaborative geoprocessing models have become one of the major solutions to significantly enhance the capacity to derive knowledge over a network, which are critical for the support of comprehensive analyses in a virtual geographic environment (VGE). With the emergence and growing maturity of the cloud computing infrastructure, a cloud-based platform for collaborative geoprocessing models promises to provide a pattern for the next generation of geoprocessing collaboration in the GIS realm. However, the problems with the existing collaborative geoprocessing models remain numerous, including the following: heterogeneity in description specifications hinders different geoprocessing services in collaborative work; the heterogeneity in messages mechanisms makes the cooperation among the geoprocessing services difficult and an integrated geoprocessing model framework centring on the collaborative model’s lifecycle is absent. To address these problems, this article proposes a cloud-based framework for building, executing and sharing collaborative models called GeoSquare: (1) a lifecycle model was designed for convenient and flexible collaborative geoprocessing; (2) a collaboration mechanism was implemented to solve specification heterogeneity; (3) a collaboration method and its proxy were used to resolve the heterogeneity in message communication and (4) to acquire better scalability, some elastic cloud features were utilized in the framework. A GeoSquare prototype was implemented on the Microsoft Azure Cloud to demonstrate the applicability and availability. Results show that users can build, execute, publish and share collaborative geoprocessing models with high efficiency in GeoSquare. GeoSquare provides a novel collaborative geoprocessing pattern enabling further geographic research in a cloud infrastructure.