Jae Gi Son

Parallel Job Processing Technique for Real-time Big-Data Processing Framework

Since the introduction of big data, numerous researches aiming to improve the accuracy and speed of data processing has been conducted. Many platforms that can process real-time data were developed for this purpose. Most standard data processing platforms used Spark Streaming as data analysis layer. However, its limitation in performance calls for a better alternative. This paper introduces a new data processing framework, Squall. Squall utilizes parallel processing and allows real-time data processing using streaming modules. Go was used for development. Through various experiments, the performance of our newly developed framework on processing real-time data was compared to the performance of the previously existing framework completing the same task. Results show quantitative evidence that Squall excel the platforms that use Spark Streaming. Our future work includes making modifications that will improve Squalls performance.

An IEEE1394-based real-time distributed IPC system for collaborating TMO's

The TMO (time-triggered message-triggered object) model is a well-known real-time object model for distributed timeliness computing. In a couple of years ago, we developed a Linux-based real-time kernel, named TMO-Linux, supporting deadline driven executions of TMOs. TMO-Linux and its distributed IPC subsystem have been used well in developing networked control systems consisting of cooperating embedded devices, but there have difficulties in executing some TMO applications accurately due to the lack of timeliness in distributed communications. To overcome this problem, we newly developed a real-time distributed IPC over IEEE1394 for the TMO-Linux kernel. In the new system, predictable delivery services for real-time messages are provided by isochronous transmissions of IEEE1394. To implement predictable delivery services, each node is set to have its own isochronous channel for receiving data that is allocated to a fixed time-slot bandwidth in an IEEE1394 frame. This paper presents an implementation technique for the IEEE1394-based real-time distributed IPC and collaborations of computing nodes using TMO-Linux

Squall: Stream Processing and Analysis Model Design

Recently, the importance of velocity, one of the characteristics of big data (5V: Volume, Variety, Velocity, Veracity, and Value), has been emphasized in the data processing, which has led to several studies on the real-time stream processing, a technology for quick and accurate processing and analyses of big data. In this study, we propose a Squall framework using in-memory technology. Moreover, we provide a description of Squall framework and its operations. This Squall framework can support the real-time event stream processing and micro-batch processing, showing high performance and memory efficiency for stream processing using Gos excellent concurrency and GC (Garbage Collection) available without a virtual machine. Therefore, you can run many jobs on one machine. In addition, the data flows through the memory, the number of operation steps are incorporated to improve the performance. It provides relatively good performance compared to existing Apache Storm and spark streaming. In conclusion, it can be used as a general-purpose big data processing framework because it can overcome the drawbacks of existing Apache storm or Spark streaming by introducing the advantages of Go language.

Block I/O Accelerator Technique to Improve Data Access Performance based Linux Multiple Disk

The Linux-based legacy server systems are configured and used with software RAID to improve the performance of the disk I/O. However, the problem is that the current Linux kernel and software RAID are difficult to optimize the high-performance block I/O because it is designed to be optimized for low-speed devices. Therefore, we propose the efficient method using recombination and re-mapping techniques to improve the performance of Block I/O Accelerator based software RAID level-0 provided on the Linux kernel level. This proposed method is designed to have more bandwidth at a time by reducing the number of system calls considering the block I/O characteristics of Linux kernel and RAID level 0. As a low-level I/O benchmarking tool, XDD is used to evaluate the performance of the proposed method. According to the experimental results, our performance gains are 15.24% on write bandwidth and 14.87% on read bandwidth compared with legacy software RAID 0.