Hyosu Kim

Mobile maestro: enabling immersive multi-speaker audio applications on commodity mobile devices

The goal of this work is to provide an abstraction of ideal sound environments to a new emerging class of Mobile Multi-speaker Audio (MMA) applications. Typically, it is challenging for MMA applications to implement advanced sound features (e.g., surround sound) accurately in mobile environments, especially due to unknown, irregular loudspeaker configurations. Towards an illusion that MMA applications run over specific loudspeaker configurations (i.e., speaker type, layout), this work proposes AMAC, a new Adaptive Mobile Audio Coordination system that senses the acoustic characteristics of mobile environments and controls individual loud-speakers adaptively and accurately. The prototype of AMAC implemented on commodity smartphones shows that it provides the coordination accuracy in sound arrival time in several tens of microseconds and reduces the variance in sound level substantially.

Rethinking Energy-Performance Trade-Off in Mobile Web Page Loading

Web browsing is a key application on mobile devices. However, mobile browsers are largely optimized for performance, imposing a significant burden on power-hungry mobile devices. In this work, we aim to reduce the energy consumed to load web pages on smartphones, preferably without increasing page load time and compromising user experience. To this end, we first study the internals of web page loading on smartphones and identify its energy-inefficient behaviors. Based on our findings, we then derive general design principles for energy-efficient web page loading, and apply these principles to the open-source Chromium browser and implement our techniques on commercial smartphones. Experimental results show that our techniques are able to achieve a 24.4% average system energy saving for Chromium on a latest-generation big.LITTLE smartphone using WiFi (a 22.5% saving when using 3G), while not increasing average page load time. We also show that our proposed techniques can bring a 10.5% system energy saving on average with a small 1.69\% increase in page load time for mobile Firefox web browser. User study results indicate that such a small increase in page load time is hardly perceivable.

Aciom: application characteristics-aware disk and network i/o management on android platform

The last several years have seen a rapid increase in smart phone use. Android offers an open-source software platform on smart phones, that includes a Linux-based kernel, Java applications, and middleware. The Android middleware provides system libraries and services to facilitate the development of performance-sensitive or device-specific functionalities, such as screen display, multimedia, and web browsing. Android keeps track of which applications make use of which system services for some pre-defined functionalities, and which application is running in the foreground attracting the users attention. Such information is valuable in capturing application characteristics and can be useful for resource management tailored to application requirements. However, the Linux-based Android kernel does not utilize such information for I/O resource management. This paper is the first work, to the best of our knowledge, to attempt to understand application characteristics through Android architecture and to incorporate those characteristics into disk and network I/O management. Our proposed approach, Aciom (Application Characteristics-aware I/O Management), requires no modification to applications and characterizes application I/O requests as time-sensitive, bursty, or plain, depending on which system services are involved and which application receives the users focus. Aciom then provides differentiated I/O management services for different types of I/O requests, supporting minimum bandwidth reservations for time-sensitive requests and placing maximum bandwidth limits on bursty requests. We present the design of Aciom and a prototype implementation on Android. Our experimental results show that Aciom is quite effective in handling disk and network I/O requests in support of time-sensitive applications in the presence of bursty I/O requests.

GPU-SAM

We examine benefits and costs of split-and-merge execution on multi-GPU systems.The split-and-merge execution can improve schedulability on real-time systems.We model schedulability analysis for split-and-merge execution.We propose an algorithm called GPA, to decide the number of GPUs to be used.We demonstrate through evaluations that GPA can improve system-wide schedulability. Multi-GPUs appear as an attractive platform to speed up data-parallel GPGPU computation. The idea of split-and-merge execution has been introduced to accelerate the parallelism of multiple GPUs even further. However, it has not been explored before how to exploit such an idea for real-time multi-GPU systems properly. This paper presents an open-source real-time multi-GPU scheduling framework, called GPU-SAM, that transparently splits each GPGPU application into smaller computation units and executes them in parallel across multiple GPUs, aiming to satisfy real-time constraints. Multi-GPU split-and-merge execution offers the potential for reducing an overall execution time but at the same time brings various different influences on the schedulability of individual applications. Thereby, we analyze the benefit and cost of split-and-merge execution on multiple GPUs and derive schedulability analysis capturing seemingly conflicting influences. We also propose a GPU parallelism assignment policy that determines the multi-GPU mode of each application from the perspective of system-wide schedulability. Our experiment results show that GPU-SAM is able to improve schedulability in real-time multi-GPU systems by relaxing the restriction of launching a kernel on a single GPU only and choosing better multi-GPU execution modes.

SounDroid: Supporting Real-Time Sound Applications on Commodity Mobile Devices

A variety of advantages from sounds such as measurement and accessibility introduces a new opportunity for mobile applications to offer broad types of interesting, valuable functionalities, supporting a richer user experience. However, in spite of the growing interests on mobile sound applications, few or no works have been done in focusing on managing an audio device effectively. More specifically, their low level of real-time capability for audio resources makes it challenging to satisfy tight timing requirements of mobile sound applications, e.g., a high sensing rate of acoustic sensing applications. To address this problem, this work presents the SounDroid framework, an audio device management framework for real-time audio requests from mobile sound applications. The design of SounDroid is based on the requirement analysis of audio requests as well as an understanding of the audio playback procedure including the audio request scheduling and dispatching on Android. It then incorporates both real-time audio request scheduling algorithms, called EDF-V and AFDS, and dispatching optimization techniques into mobile platforms, and thus improves the quality-of-service of mobile sound applications. Our experimental results with the prototype implementation of SounDroid demonstrate that it is able to enhance scheduling performance for audio requests, compared to traditional mechanisms (by up to 40% of improvement), while allowing deterministic dispatching latency.

UbiTap: Leveraging Acoustic Dispersion for Ubiquitous Touch Interface on Solid Surfaces

With the omnipresence of computing devices in our daily lives, interests in ubiquitous computing interfaces have grown. In response to this, various studies have introduced on-surface input techniques which use the surfaces of surrounding objects as a touch interface. However, these methods are yet struggling to support ubiquitous interaction due to their dependency on specific hardware or environments. In this paper, we propose UbiTap, an input method that turns solid surfaces into a touch input space, through the use of sound (i.e., with microphones already present in the commodity devices). More specifically, we develop a novel touch localization technique which leverages the physical phenomenon, referred to as dispersion, a characteristic of sound as it travels through solid surfaces, so as to address challenges which limit existing acoustic-based solutions in terms of portability, accuracy, usability, robustness, and responsiveness. Our extensive experiments with a prototype of UbiTap show that we can support sub-centimeter accuracy on various surfaces with minor user calibration effort. In our experience with real-world users, UbiTap significantly improves usability and robustness, thus enabling the emergence of more exciting applications.

Poster: TapSnoop -- Inferring Tapstrokes from Listening to Tap Sound on Mobile Devices

Mobile device users tap a touch-screen for entering sensitive information such as passwords and PIN numbers, and many works have proposed an attack model snooping such tapstrokes especially with the use of built-in sensors [1, 2, 3]. These studies raise the serious security concerns with the following attack scenario. A malicious application runs in the foreground as a normal chatting application, collecting a training set of sensor data generated from tapstrokes. While a user types her credit card number for purchasing something on a shopping application, it sneakingly takes sensor streams in the background and infers the tapped number by comparing the streams with the training data. However, in practice, the existing works have shown a limited inference accuracy, due to the following reasons. First, the intensity of tapstrokes is typically much low, resulting in a subtle change on sensor data. Second, mobile devices generally come with small on-screen keyboards where keys are very close to each other. Thus, it is essential to perform fine-grained tapstroke localization. Third, each mobile device has its own hardware characteristics with regard to screen’s size and thickness, as well as built-in sensor’s sensitivity. This inherently leads to different characteristics of tapstrokes for different devices. Last, smartphone users can use their devices in various places with different noise levels, while moving around. Therefore, it should be able to infer tapstrokes robustly against the environmental changes.

Priority-Based Network Interrupt Scheduling for Predictable Real-Time Support

Interrupt handling is generally separated from process scheduling. This can lead to a scheduling anomaly and priority inversion. The processor can interrupt a higher priority process that is currently executing, in order to handle a network packet reception interruption on behalf of its intended lower priority receiver process. We propose a new network interrupt handling scheme that combines interrupt handling with process scheduling and the priority of the process. The proposed scheme employs techniques to identify the intended receiver process of an incoming packet at an earlier phase. We implement a prototype system of the proposed scheme on Linux 2.6, and our experiment results show that the prototype system supports the predictable real-time behavior of higher priority processes even when excessive traffic is sent to lower priority processes. Category: Embedded computing; Smart and intelligent computing