Ze Deng

Parallel Processing of Dynamic Continuous Queries over Streaming Data Flows

More and more real-time applications need to handle dynamic continuous queries over streaming data of high density. Conventional data and query indexing approaches generally do not apply for excessive costs in either maintenance or space. Aiming at these problems, this study first proposes a new indexing structure by fusing an adaptive cell and KDB-tree, namely CKDB-tree. A cell-tree indexing approach has been developed on the basis of the CKDB-tree that supports dynamic continuous queries. The approach significantly reduces the space costs and scales well with the increasing data size. Towards providing a scalable solution to filtering massive steaming data, this study has explored the feasibility to utilize the contemporary general-purpose computing on the graphics processing unit (GPGPU). The CKDB-tree-based approach has been extended to operate on both the CPU (host) and the GPU (device). The GPGPU-aided approach performs query indexing on the host while perform streaming data filtering on the device in a massively parallel manner. The two heterogeneous tasks execute in parallel and the latency of streaming data transfer between the host and the device is hidden. The experimental results indicate that (1) CKDB-tree can reduce the space cost comparing to the cell-based indexing structure by 60 percent on average, (2) the approach upon the CKDB-tree outperforms the traditional counterparts upon the KDB-tree by 66, 75 and 79 percent in average for uniform, skewed and hyper-skewed data in terms of update costs, and (3) the GPGPU-aided approach greatly improves the approach upon the CKDB-tree with the support of only a single Kepler GPU, and it provides real-time filtering of streaming data with 2.5M data tuples per second. The massively parallel computing technology exhibits great potentials in streaming data monitoring.

Modeling and simulation for natural disaster contingency planning driven by high-resolution remote sensing images

Natural disasters occur unexpectedly and usually result in huge losses of life and property. How to effectively make contingency plans is an intriguing question constantly faced by governments and experts. Human rescue operations are the most critical issue in contingency planning. A natural disaster scenario is, in general, highly complicated and dynamic. Modeling and simulation technologies have been gaining considerable momentum in investigating natural disaster scenarios to enable contingency planning. However, existing MS and (2) the absence of methods and platforms to describe the collective behaviors of people in disaster situations. Considering these problems, an M&S framework for human rescue operations in a typical natural disaster, i.e., a landslide, has been developed in this study. The framework consists of three modules: (1) remote sensing information extraction, (2) landslide simulation, and (3)crowd simulation. The crowd simulation module is driven by the real/virtual data provided by the former modules. A number of simulations (using the Zhouqu landslide as an example) have been performed to study human relief operations spontaneously and under manipulation, with the effect of contingency plans highlighted. The experimental results demonstrate that (1) the simulation framework is an effective tool for contingency planning, and (2) real data can make the simulation outputs more meaningful. We enable evaluation of contingency plans using modelling and simulation technology.We model crowd behaviors under natural disasters.We develop a DDDAS simulation framework with the support of high resolution remote sensing information.

DDDAS-Based Parallel Simulation of Threat Management for Urban Water Distribution Systems

The Contaminant Source Characterization (CSC) problem in a Water Distributed System (WDS) exhibits a compute-intensive challenge that requires highly reliable and high performance computing resources in order to achieve near real-time processing. Traditional solution to the CSC problem with MPI via Grid/cluster computing cannot fulfill CSC’s QoS requirements, such as, reliability, scalability and flexibility. To address the aforementioned research issues, we have developed a parallel solution to the CSC problem using MapReduce in Clouds, which mainly includes 1) parallelization of the process of evaluating individuals in the Genetic Algorithm for CSC with MapReduce, and 2) developing an advanced cyberinfrastructure in an academic Cloud computing test bed (the FutureGrid test bed). We have carried out performance evaluation and discussion on our solution. Test results and performance evaluation show that parallel GA with MapReduce in a dynamic cyberinfrastructure can deliver a high performance, fault tolerance and flexible solution for the CSC problem.

Review: Large scale distributed visualization on computational Grids: A review

Advances in science and engineering have put high demands on tools for high performance large-scale data exploration and analysis. Visualization is a powerful technology for analyzing data and presenting results. Todays science and engineering have benefited from state-of-the-art of Grid technologies and modern visualization systems. To visualize the large amount of data, rendering technologies are widely used to parallelize visualization tasks over distributed resources on computational Grids. It raises the necessity to balance the computational load and to minimize the network bandwidth requirements. This article explains in Grid environments how new approaches of visualization architecture and load-balancing algorithms address these challenges in a principled fashion. The Grid infrastructure that supports large scale distributed visualization is also introduced. Some typical visualization systems on Grids are referenced for discussions.

An efficient online direction-preserving compression approach for trajectory streaming data

Online trajectory compression is an important method of efficiently managing massive volumes of trajectory streaming data. Current online trajectory methods generally do not preserve direction information and lack high computing performance for the fast compression. Aiming to solve these problems, this paper first proposed an online direction-preserving simplification method for trajectory streaming data, online DPTS by modifying an offline direction-preserving trajectory simplification (DPTS) method. We further proposed an optimized version of online DPTS called online DPTS+ by employing a data structure called bound quadrant system (BQS) to reduce the compression time of online DPTS. To provide a more efficient solution to reduce compression time, this paper explored the feasibility of using contemporary general-purpose computing on a graphics processing unit (GPU). The GPU-aided approach paralleled the major computing part of online DPTS+ that is the SP-theo algorithm. The results show that by maintaining a comparable compression error and compression rate, (1) the online DPTS outperform offline DPTS with up to 21% compression time, (2) the compression time of online DPTS+ algorithm is 3.95 times faster than that of online DPTS, and (3) the GPU-aided method can significantly reduce the time for graph construction and for finding the shortest path with a speedup of 31.4 and 7.88 (on average), respectively. The current approach provides a new tool for fast online trajectory streaming data compression.

Massively parallel non-stationary EEG data processing on GPGPU platforms with Morlet continuous wavelet transform

Morlet continuous wavelet transform (MCWT) has been widely used to process non-stationary electro-encephalogram (EEG) data. Nowadays, the MCWT application for processing EEG data is time-sensitive and data-intensive due to quickly increasing problem domain sizes and advancing experimental techniques. In this paper, we proposed a massively parallel MCWT approach based on GPGPU to address this research challenge. The proposed approach treats MCWT as four main computing sub-procedures and parallelizes them with CUDA correspondingly. We focused on optimizing FFT on GPUs to improve the performance of MCWT. Extensive experiments have been carried out on Fermi and Kepler GPUs and a Fermi GPU cluster. The results indicate that (1) the proposed approach (especially on Kepler GPU) can ensure encouraging runtime performance of processing non-stationary EEG data in contrast to CPU-based MCWT, (2) the performance can further be improved on the GPU cluster but performance bottleneck exists when running multiple GPGPUs on one node, and (3) tuning an appropriate FFT radix is important to the performance of our MCWT.

Variation-Aware Layer Assignment With Hierarchical Stochastic Optimization on a Multicore Platform

As the very large scale integration (VLSI) technology enters the nanoscale regime, VLSI design is increasingly sensitive to variations on process, voltage, and temperature. Layer assignment technology plays a crucial role in industrial VLSI design flow. However, existing layer assignment approaches have largely ignored these variations, which can lead to significant timing violations. To address this issue, a variation-aware layer assignment approach for cost minimization is proposed in this paper. The proposed layer assignment approach is a single-stage stochastic program that directly controls the timing yield via a single parameter, and it is solved using Monte Carlo simulations and the Latin hypercube sampling technique. A hierarchical design is also adopted to enable the optimization process on a multicore platform. Experiments have been performed on 5000 industrial nets, and the results demonstrate that the proposed approach: 1) can significantly improve the timing yield by 64% in comparison with the nominal design and 2) can reduce the wire cost by 15.7% in comparison with the worst case design.

A Simulation Study on Urban Water Threat Detection in Modern Cyberinfrastructures

The computation of Contaminant Source Characterization (CSC) is a critical research issue in Water Distribution System (WDS) management. We use a simulation framework to identify optimized locations of sensors that lead to fast detection of contamination sources. The optimization engine is based on a Genetic Algorithm (GA) that interprets trial solutions as individuals. During the optimization process many thousands of these solutions are generated. For a large WDS, the calculation of these solutions are non-trivial and time consuming. Hence, it is a compute intensive application that requires significant compute resources. Furthermore, we strive to generate solutions quickly in order to respond to the urgency of a response. To carry out the calculations we require user-level middleware that can be supporting the workflow of the application and manages the resource assignment in an efficient and fault tolerant fashion. To do so we have prototyped the middleware framework that provides a convenient command line and portal layer of steering applications on Grids. Internally, we utilize a sophisticated workflow engine that provides the ability to access elementary fault tolerant mechanisms for job scheduling. This includes the management of job replicas and the reaction on late return of results. We report the test results of CSC problem solving on a real Grid test bed - the Tera Grid test bed. In addition, we contrast this system architecture with a Hadoop-based implementation that automatically includes fault tolerance. The later activity has been conducted on Future Grid.

Confrontation Scenario Simulation Using Functional Programming Model

Simulation is an important approach to the study of scenarios of confrontation among antagonistic groups. It remains a research issue to explore the influence of the crowd size and imbalance between groups’ sizes on the process of confrontation. In this study, a multi-agent simulation system has been developed with functional programming model (FPM). FPM can easily formulate the simultaneous behaviors/actions of individuals. It also provides “communication backbone” for agents’ interactions. A timing system has been designed to drive the simulation procedure. Our simulations focus on whether/how a confrontational scenario may remain stable. Experimental results indicate that with the increment of the overall size of the crowd in confrontation, the possibility of the scenario getting out of control rises. A relatively small scale of crowd is much more controllable.