Sung Jin Hur

Load adaptive and fault tolerant distributed stream processing system for explosive stream data

As smart devices such as sensors, smartphones, and CCTVs are becoming extensively utilized recently, stream data from those smart devices are consistently generated explosively. There are also increasing cases that we notice security attacks after already important assets are damaged by cyber-targeted attacks such as APT attacks due to the lack of real-time security log processing capability. Accordingly, the demand to process and analyse the exploding stream data in real-time and in advance is consistently increasing in many application domains. However, existing distributed stream processing systems like Storm and S4 are not well adaptive when there are drastic increase of input stream data. In this paper, we propose a distributed stream processing system which supports several load adaptation techniques utilizable for various circumstances of explosive data stream, and also supports fault tolerance mechanisms to fail over in several failure situations.

Performance evaluation of software streaming server architecture for massive users

In this paper, we propose the streaming server architecture for massive users and verify it by a small scale simulation with virtual users. Software streaming is one of the ASP solution that allows the execution of stream-enabled software, even while the transmission of the program may still be in progress. In this paper, we analyze workload characteristics of the software stream, and extract four requirements that software streaming system have to satisfy. Based on these requirements, we design the software streaming server architecture. The verification of the designed server is performed by virtual user simulation. The result tells us the capacity of each server and helps us to calculate the scale of the system for massive users

Load adaptive distributed stream processing system for explosive stream data

As smart devices such as sensors, smartphones, and CCTVs are becoming extensively utilized recently, stream data from those smart devices are consistently generated explosively. There are also increasing cases that we notice security attacks after already important assets are damaged by cyber-targeted attacks such as APT attacks due to the lack of real-time security log processing capability. Accordingly, the demand to process and analyse the exploding stream data in real-time and in advance is consistently increasing in many application domains. However, existing distributed stream processing systems like Storm and S4 are not well adaptive when there are drastic increase of input stream data. In this paper, we propose a distributed stream processing system which supports several load adaptation techniques utilizable for various circumstances of explosive data stream.

Symbiote Coprocessor Unit—A Streaming Coprocessor for Data Stream Acceleration

This paper describes the design and the architecture of symbiote coprocessor unit (SCU)-a programmable streaming coprocessor for a heterogeneous reconfigurable logic-assisted data stream management systems (DSMSs) such as symbiote. The SCU is intended for streaming applications with real-time event and data processing that have stricter deadlines, high-bandwidth, and high-accuracy requirements. To meet these requirements, the SCU exploits unique characteristics of DSMSs, such as single-pass tuple processing, windowed operators, and inherent data level parallelism, using a single-instruction multiple-data very large instruction word (SIMD-VLIW) microarchitecture and a novel inverted distributed register file. In order to better explain the instruction set, design, and functionality of the various units in the SCU, this paper also provides a brief overview of SymQL-a procedural query language that we developed to describe the queries that can be executed on the SCU. Finally, this paper presents the performance of SCU using four queries that represent common data stream processing use-cases, one of them being similar to a query found in the Linear Road Benchmark. Using these queries on SCU simulation, we show that the SCU outperforms a software-only DSMS running on an AMD Opteron 2350 quad-core processor by 1.5-42 times.

Design of a scalable data stream channel for big data processing

This paper outlines big data infrastructure for processing data streams. Our project is distributed stream computing platform that provides cost-effective and large-scale big data services by developing data stream management system. This research contributes to advancing feasibility of big data processing for distributed, real-time computation even when they are overloaded.

Analysis of Software Streaming Data

In this paper, we analyze the software streaming data for stable service and efficient resource allocation of the server. We introduce the software streaming service and the message flow of the service. Based on the message flow, we firstly divide the software stream into the initial phase and the on-demand phase. And we further divide the initial phase into the downloading period and the processing period. Finally, we classify software into the downloading based software and the processing based software according to the analysis of the proportion of each period. And we also define two external environmental factors (current available network bandwidth between server and client and the processing power of the client PC) which could affect overall performance of the software streaming service. We demonstrate the external environment that affects the ALT and SRU. So we validate it by several experiments. The result shows that the streaming server consumes the most resource when the rich client uses the downloading based software in the high speed network, and there is an inverse proportion between the ALT and the SRM. Using all the information (software type, network condition, client processing power), the streaming server can allocate resource to a client more accurately.