Shoichi Saito

Dynamic Processing Slots Scheduling for I/O Intensive Jobs of Hadoop MapReduce

Hadoop, consists of Hadoop MapReduce and Hadoop Distributed File System (HDFS), is a platform for large scale data and processing. Distributed processing has become common as the number of data has been increasing rapidly worldwide and the scale of processes has become larger, so that Hadoop has attracted many cloud computing enterprises and technology enthusiasts. Hadoop users are expanding under this situation. Our studies are to develop the faster of executing jobs originated by Hadoop. In this paper, we propose dynamic processing slots scheduling for I/O intensive jobs of Hadoop MapReduce focusing on I/O wait during execution of jobs. Assigning more tasks to added free slots when CPU resources with the high rate of I/O wait have been detected on each active Task Tracker node leads to the improvement of CPU performance. We implemented our method on Hadoop 1.0.3, which results in an improvement of up to about 23% in the execution time.

Implementation and Evaluation of the JobTracker Initiative Task Scheduling on Hadoop

MapReduce is one of the major successful framework to process large-scale data efficiently. Distributed programs can be implemented easily by describing only two methods, Map and Reduce. In Hadoop which is an open source implementation of MapReduce, a JobTracker (master program in Hadoop) assigns Map Tasks and Reduce Tasks to TaskTrackers (slave programs which execute the tasks). In an environment that multiple Hadoops are running on a physical machine, its computational resources should be shared by every Hadoop (Multi-Hadoop environment). In this environment, available computational resources of each Hadoop fluctuate dynamically by behaviors of other Hadoops. Therefore, the JobTracker needs to decide assignment of tasks based on loads and available computation resources on the cluster (JobTracker Initiative Task Scheduler). In this paper, we propose a method which decides the number of task executions in order to use computational resources efficiently based on a load on each computer. And we evaluate its performance, and our results show that the proposal method has achieved a reduction of execution times of jobs by about 11.1% in Multi-Hadoop environment as compared to original Hadoop.

Improving Response Time for Cassandra with Query Scheduling

A management of large-scale data becomes more important, along with the spread of cloud service and the speed-up of networks. Since data management on a single machine can cause performance and scalability problems, data management across multiple machines has been proposed. Distributed Key Value Store(KVS) is a data store which manages data across multiple machines. Since distributed KVSs manage data which consists of simple key-value pair, they can achieve scalability easily. Distributed KVSs are widely used in many services managing large-scale data, such as Facebook and Twitter. Distributed KVSs provide interfaces to access key-value pair by simply specifying the key. In this paper, we refer to a query which only obtains a value from a key as a single query. Some distributed KVSs support a range query which obtains multiple values from a key range. However, under mixed query workloads that consist of single and range queries, single queries(which can be executed faster) are forced to wait until preceding range queries are finished. And this results in the increase of average response time. We propose an approach to reduce the average response time by query scheduling. We implemented our method on Cassandra, and evaluation results showed a reduction of the average response time.

Tracing malicious injected threads using alkanet Malware analyzer

Recently, malware has become a major security threat to computers. Responding to threats from malware requires malware analysis and understanding malware behavior. However, malware analysts cannot spend the time required to analyze each instance of malware because unique variants of malware emerge by the thousands every day. Dynamic analysis is effective for understanding malware behavior within a short time. The method of analysis to execute the malware and observe its behavior using debugging and monitoring tools. We are developing Alkanet, a malware analyzer that uses a virtual machine monitor based on BitVisor. Alkanet can analyze malware even if the malware applies anti-debugging techniques to thwart analysis by dynamic analysis tools. In addition, analysis overhead is reduced. Alkanet executes malware on Windows XP, and traces system calls invoked by threads. Therefore, the system can analyze malware that infects other running processes. Also, the system call logs are obtained in real time via a IEEE 1394 interface. Other programs can readily examine the log and process the analysis results to understand intentions of malware behavior. In this paper, we describe the design and implementation of Alkanet. We confirm that Alkanet analyzes malware behaviors, such as copying itself, deleting itself, and creating new processes. We also confirm that Alkanet accurately traces threads injected by malware into other processes.

Forecasting Students’ Future Academic Records Using Past Attendance Recording Data and Grade Data☆

Abstract In this study, the authors forecast students’ future academic records using past attendance recording data and grade data. We use a Bayesian network as forecasting method. During construction of the Bayesian network forecasting model, unnecessary variables become noise and so lower the forecasting accuracy. Therefore, to improve the forecasting accuracy, we used information gain to reduce the number of variables in the model. As a result, accuracy improved.

Design and Implementation of an Application State Migration Mechanism between Android Devices

Devices equipped with Android, which is the OS for mobile devices, spread widely, and one user may own plural devices. It is thought that the user uses plural devices properly depending on the situation. For example, the user may use a small device on the way, and a large device in the office. In such situation, it is necessary for improvement of work efficiency to change work environment seamlessly at plural devices. We design and implement a platform to transmit an application used now to other device. This platform enables a user to resume work with the other device. The platform acquires a state of application working at a transmission side device, and transmits it using the communication between the devices. The application in a reception side is restored by the platform using the state. Bluetooth is used for the communication. We offer the platform as a function of Android OS. In this article, we describe the implementation of the platform, the measurements of time to use it, and consideration about results of the measurements.

VSE: Virtual Switch Extension for Adaptive CPU Core Assignment in Softirq

An Edge-Overlay model constructing virtual networks using both virtual switches and IP tunnels is promising in cloud datacenter networks. But software-implemented virtual switches can cause performance problems because the packet processing load is concentrated on a particular CPU core. Although multi queue functions like Receive Side Scaling (RSS) can distribute the load onto multiple CPU cores, there are still problems to be solved such as IRQ core collision of heavy traffic flows as well as competitive resource use between physical and virtual for packet processing. In this paper, we propose a software packet processing unit named VSE (Virtual Switch Extension) to address these problems by adaptively determining softirq cores based on both CPU load and VM-running information. Furthermore, the behavior of VSE can be managed by Open Flow controllers. Our performance evaluation results showed that throughput of our approach was higher than an existing RSSbased model as packet processing load increased. In addition, we show that our method prevented performance of high-loaded flows from being degraded by priority-based CPU core selection.

Orthros: A High-Reliability Operating System with Transmigration of Processes

We propose a method to solve problems that accompany recovering from operating system (OS) failures. First, to reduce recovery time, we make two OSes run simultaneously and configure them as an active-backup structure in one computer. This structure can provide a fast recovery from failures by a failover. Recovery time when using the proposed method is about 0.4 seconds at a minimum and up to about 10 seconds even if 2 GB memory is restored. Next, for smooth continuation of services after recovery, the proposed method preserves processes, their network connections, and file caches, and does not have runtime overhead to obtain a process execution status from the running active OS before a crash. In addition, the resources consumed to build the active-backup structure are only one CPU core and a small amount of memory. The hardware required to implement the proposed method is a multi-core processor and one disk for each OS, consequently, introduction of the proposed method incurs low cost. In the evaluation, we confirmed that the downtime was up to about 1.5 seconds when the active OS of the proposed system crashed while running a text editor, an NFS server, and a database server.

Monitoring Library Function-based Intrusion Prevention System with Continuing Execution Mechanism

Anomaly-based Intrusion Prevention Systems have been studied to prevent zero-day attacks. However these existing systems can

Bifrost : A Novel Anonymous Communication System with DHT

B!G (Bt prevent mimicry attacks because of the inadequacy of monitoring accuracy. Moreover, they provide no continuity for monitored applications when they have been compromised. In this paper, we propose a novel Intrusion Prevention System named Belem that detects anomaly states by checking the ordering of library functions and has a Continuing Execution Mechanism to provide application continuity. We implemented Belem on Linux and evaluated it.