Jizhe Xia

Evaluating open-source cloud computing solutions for geosciences

Many organizations start to adopt cloud computing for better utilizing computing resources by taking advantage of its scalability, cost reduction, and easy to access characteristics. Many private or community cloud computing platforms are being built using open-source cloud solutions. However, little has been done to systematically compare and evaluate the features and performance of open-source solutions in supporting Geosciences. This paper provides a comprehensive study of three open-source cloud solutions, including OpenNebula, Eucalyptus, and CloudStack. We compared a variety of features, capabilities, technologies and performances including: (1) general features and supported services for cloud resource creation and management, (2) advanced capabilities for networking and security, and (3) the performance of the cloud solutions in provisioning and operating the cloud resources as well as the performance of virtual machines initiated and managed by the cloud solutions in supporting selected geoscience applications. Our study found that: (1) no significant performance differences in central processing unit (CPU), memory and I/O of virtual machines created and managed by different solutions, (2) OpenNebula has the fastest internal network while both Eucalyptus and CloudStack have better virtual machine isolation and security strategies, (3) Cloudstack has the fastest operations in handling virtual machines, images, snapshots, volumes and networking, followed by OpenNebula, and (4) the selected cloud computing solutions are capable for supporting concurrent intensive web applications, computing intensive applications, and small-scale model simulations without intensive data communication.

Using adaptively coupled models and high-performance computing for enabling the computability of dust storm forecasting

Forecasting dust storms for large geographical areas with high resolution poses great challenges for scientific and computational research. Limitations of computing power and the scalability of parallel systems preclude an immediate solution to such challenges. This article reports our research on using adaptively coupled models to resolve the computational challenges and enable the computability of dust storm forecasting by dividing the large geographical domain into multiple subdomains based on spatiotemporal distributions of the dust storm. A dust storm model (Eta-8bin) performs a quick forecasting with low resolution (22 km) to identify potential hotspots with high dust concentration. A finer model, non-hydrostatic mesoscale model (NMM-dust) performs high-resolution (3 km) forecasting over the much smaller hotspots in parallel to reduce computational requirements and computing time. We also adopted spatiotemporal principles among computing resources and subdomains to optimize parallel systems and improve the performance of high-resolution NMM-dust model. This research enabled the computability of high-resolution, large-area dust storm forecasting using the adaptively coupled execution of the two models Eta-8bin and NMM-dust.

A Service Brokering and Recommendation Mechanism for Better Selecting Cloud Services

Cloud computing is becoming the new generation computing infrastructure, and many cloud vendors provide different types of cloud services. How to choose the best cloud services for specific applications is very challenging. Addressing this challenge requires balancing multiple factors, such as business demands, technologies, policies and preferences in addition to the computing requirements. This paper recommends a mechanism for selecting the best public cloud service at the levels of Infrastructure as a Service (IaaS) and Platform as a Service (PaaS). A systematic framework and associated workflow include cloud service filtration, solution generation, evaluation, and selection of public cloud services. Specifically, we propose the following: a hierarchical information model for integrating heterogeneous cloud information from different providers and a corresponding cloud information collecting mechanism; a cloud service classification model for categorizing and filtering cloud services and an application requirement schema for providing rules for creating application-specific configuration solutions; and a preference-aware solution evaluation mode for evaluating and recommending solutions according to the preferences of application providers. To test the proposed framework and methodologies, a cloud service advisory tool prototype was developed after which relevant experiments were conducted. The results show that the proposed system collects/updates/records the cloud information from multiple mainstream public cloud services in real-time, generates feasible cloud configuration solutions according to user specifications and acceptable cost predication, assesses solutions from multiple aspects (e.g., computing capability, potential cost and Service Level Agreement, SLA) and offers rational recommendations based on user preferences and practical cloud provisioning; and visually presents and compares solutions through an interactive web Graphical User Interface (GUI).

A performance, semantic and service quality-enhanced distributed search engine for improving geospatial resource discovery

Geospatial resource discovery is a critical step for developing geographic science applications. With the increasing number of geospatial resources available online, many Spatial Data Infrastructure (SDI) components (e.g. catalogues and portals) have been developed to help manage and discover geospatial resources. However, efficient and accurate geospatial resource discovery is still a big challenge because of the heterogeneity and complexity of decentralized network environments and interdisciplinary semantics. In this article, we report a search engine framework for efficient geospatial resource discovery, which reduces integration costs by leveraging existing Geospatial Cyberinfrastructure (GCI) components. Specifically, (1) the framework provides integration capability and flexibility by adopting the brokering approach, implementing a ‘plug-in’-based framework for metadata processing and proposing a dynamically configurable search workflow; (2) the asynchronous messaging and batch processing-based metadata record retrieval mode enhances the search performance and user interactivity; (3) an embedded semantic support system improves the discovery recall level and precision by providing semantic-based search rule creation and result similarity evaluation functions and (4) the engine assists user decision-making by integrating a service quality monitoring and evaluation system, data/service visualization tools, multiple views and additional information. Experiments and a search example show that the proposed engine helps both scientists and general users search for more accurate results with enhanced performance and user experience through a user-friendly interface.

A New Model for a Carpool Matching Service

Carpooling is an effective means of reducing traffic. A carpool team shares a vehicle for their commute, which reduces the number of vehicles on the road during rush hour periods. Carpooling is officially sanctioned by most governments, and is supported by the construction of high-occupancy vehicle lanes. A number of carpooling services have been designed in order to match commuters into carpool teams, but it known that the determination of optimal carpool teams is a combinatorially complex problem, and therefore technological solutions are difficult to achieve. In this paper, a model for carpool matching services is proposed, and both optimal and heuristic approaches are tested to find solutions for that model. The results show that different solution approaches are preferred over different ranges of problem instances. Most importantly, it is demonstrated that a new formulation and associated solution procedures can permit the determination of optimal carpool teams and routes. An instantiation of the model is presented (using the street network of Guangzhou city, China) to demonstrate how carpool teams can be determined.

Adopting cloud computing to optimize spatial web portals for better performance to support Digital Earth and other global geospatial initiatives

A spatial web portal (SWP) provides a web-based gateway to discover, access, manage, and integrate worldwide geospatial resources through the Internet and has the access characteristics of regional to global interest and spiking. Although various technologies have been adopted to improve SWP performance, enabling high-speed resource access for global users to better support Digital Earth remains challenging because of the computing and communication intensities in the SWP operation and the dynamic distribution of end users. This paper proposes a cloud-enabled framework for high-speed SWP access by leveraging elastic resource pooling, dynamic workload balancing, and global deployment. Experimental results demonstrate that the new SWP framework outperforms the traditional computing infrastructure and better supports users of a global system such as Digital Earth. Reported methodologies and framework can be adopted to support operational geospatial systems, such as monitoring national geographic state and spanning across regional and global geographic extent.

Optimizing an index with spatiotemporal patterns to support GEOSS Clearinghouse

A variety of Earth observation systems monitor the Earth and provide petabytes of geospatial data to decision-makers and scientists on a daily basis. However, few studies utilize spatiotemporal patterns to optimize the management of the Big Data. This article reports a new indexing mechanism with spatiotemporal patterns integrated to support Big Earth Observation (EO) metadata indexing for global user access. Specifically, the predefined multiple indices mechanism (PMIM) categorizes heterogeneous user queries based on spatiotemporal patterns, and multiple indices are predefined for various user categories. A new indexing structure, the Access Possibility R-tree (APR-tree), is proposed to build an R-tree-based index using spatiotemporal query patterns. The proposed indexing mechanism was compared with the classic R*-tree index in a number of scenarios. The experimental result shows that the proposed indexing mechanism generally outperforms a regular R*-tree and supports better operation of Global Earth Observation System of Systems (GEOSS) Clearinghouse.

Contemporary Computing Technologies for Processing Big Spatiotemporal Data

Geographic phenomena evolve in a four-dimensional spatiotemporal world. To capture the geographical phenomena at different scales, large amount of data (big data) are produced with specific spatiotemporal patterns. Phenomena evolution and the principles driving the evolution provide pathways for developing methodological solutions to process the big spatiotemporal data. Based on experiences gained from several projects, such as climate studies and cloud computing, we introduce in this chapter modern computing technologies required for processing big data, including (1) sensor web, Earth observations, and model simulations for collecting and generating big data, (2) flexible and standard-based systems for managing big data for easy discovery and access, (3) multidimensional visual analytics for exploring and analyzing big spatiotemporal data, and (4) grid, cloud, and GPU computing for addressing the computing intensive challenges. We discuss through exemplar projects how these cutting-edge computing technologies are utilized to handle big spatiotemporal data. We expect this chapter to set a computing research context for future big data handling at different spatiotemporal granules.

Forming a global monitoring mechanism and a spatiotemporal performance model for geospatial services

Geographic information service (GIService) has become popular in the last decade to develop applications for addressing global challenges. Performance is one of the most important criteria to help users select distributed online GIService for developing geospatial applications including natural hazards and emergency responses. However, performance accuracy is limited by the single-location-based evaluation mechanism while service performance is dynamic in space and time between end-users and services. We propose a spatiotemporal performance evaluation mechanism to improve the accuracy. Specially, a cloud and volunteer computing mechanism is proposed to collect performance information of globally distributed GIServices. A global spatiotemporal performance model is designed to integrate spatiotemporal dynamics for better performance evaluation for users from different regions at different times. This model is tested to support GIService selection in global spatial data infrastructures (SDIs). The experiment confirms that the proposed model provides more accurate evaluations for global users and better supports geospatial resource utilizations in SDIs than previous mechanisms. The methodology can be adopted to improve the services of other regional and global distributed operational systems.

An experimental study of open-source cloud platforms for dust storm forecasting

Cloud computing is becoming a viable computing solution for scientific research and several open-source cloud solutions are available to support scientific studies. However, little has been done to systematically investigate the performance of these solutions in supporting scientific pursuits. Taking dust storm forecasting as an example, we test three popular open-source cloud solutions, namely Eucalyptus, OpenNebula, and CloudStack, on the same hardware and compare against a bare cluster. We find that: (1) compared to the bare cluster, a cloud has about 10% virtualization and management overhead when one virtual machine is used. Overhead increases when more virtual machines are used. Leveraging more virtual resources would not necessarily yield better performance. (2) For computing- and communication-intensive dust storm forecasting, the performance overhead is mainly due to virtualized network rather than virtualized computing resources when more than one virtual machine is involved. (3) Compared to Eucalyptus and CloudStack, OpenNebula provides better support for dust storm forecasting with relatively better performance. The results can provide some insights for scientific community in adopting these open-source cloud solutions.