Jaesik Choi

Spatio-temporal pyramid matching for sports videos

In this paper, we address the problem of querying video shots based on content-based matching. Our proposed system automatically partitions a video stream into video shots that maintain continuous movements of objects. Finding video shots of the same category is not an easy task because objects in a video shot change their locations over time. Our spatio-temporal pyramid matching (STPM) is the modified spatial pyramid matching (SPM), which considers temporal information in conjunction with spatial locations to match objects in video shots. In addition, we model the mathematical condition in which temporal information contributes to match video shots. In order to improve the matching performance, dynamic features including movements of objects are considered in addition to static features such as edges of objects. In our experiments, several methods based on different feature sets and matching methods are compared, and our spatio-temporal pyramid matching performed better than existing methods in video matching for sports videos.

Safe Navigation of a Mobile Robot Considering Visibility of Environment

We present one approach to achieve safe navigation in an indoor dynamic environment. So far, there have been various useful collision avoidance algorithms and path planning schemes. However, those algorithms possess fundamental limitations in that the robot can avoid only ldquovisiblerdquo ones among surrounded obstacles. In a real environment, it is not possible to detect all the dynamic obstacles around the robot. There are many occluded regions due to the limited field of view. In order to avoid collisions, it is desirable to exploit visibility information. This paper proposes a safe navigation scheme to reduce collision risk considering occluded dynamic obstacles. The robots motion is controlled by the hybrid control scheme. The possibility of collision is dually reflected to path planning and speed control. The proposed scheme clearly indicates the structural procedure on how to model and to exploit the risk of navigation. The proposed scheme is experimentally tested in a real office building. The experimental results show that the robot moves along the safe path to obtain sufficient field of view. In addition, safe speed constraints are applied in motion control. It is experimentally verified that a robot safely navigates in dynamic indoor environment by adopting the proposed scheme.

SecureCore: A multicore-based intrusion detection architecture for real-time embedded systems

Security violations are becoming more common in real-time systems - an area that was considered to be invulnerable in the past - as evidenced by the recent W32.Stuxnet and Duqu worms. A failure to protect such systems from malicious entities could result in significant harm to both humans as well as the environment. The increasing use of multicore architectures in such systems exacerbates the problem since shared resources on these processors increase the risk of being compromised. In this paper, we present the SecureCore framework that, coupled with novel monitoring techniques, is able to improve the security of realtime embedded systems. We aim to detect malicious activities by analyzing and observing the inherent properties of the real-time system using statistical analyses of their execution profiles. With careful analysis based on these profiles, we are able to detect malicious code execution as soon as it happens and also ensure that the physical system remains safe.

Combining planning and motion planning

Robotic manipulation is important for real, physical world applications. General Purpose manipulation with a robot (eg. delivering dishes, opening doors with a key, etc.) is demanding. It is hard because (1) objects are constrained in position and orientation, (2) many non-spatial constraints interact (or interfere) with each other, and (3) robots may have multi-degree of freedoms (DOF). In this paper we solve the problem of general purpose robotic manipulation using a novel combination of planning and motion planning. Our approach integrates motions of a robot with other (non-physical or external-to-robot) actions to achieve a goal while manipulating objects. It differs from previous, hierarchical approaches in that (a) it considers kinematic constraints in configuration space (C-space) together with constraints over object manipulations; (b) it automatically generates high-level (logical) actions from a C-space based motion planning algorithm; and (c) it decomposes a planning problem into small segments, thus reducing the complexity of planning.

Lifted relational Kalman filtering

Kalman Filtering is a computational tool with widespread applications in robotics, financial and weather forecasting, environmental engineering and defense. Given observation and state transition models, the Kalman Filter (KF) recursively estimates the state variables of a dynamic system. However, the KF requires a cubic time matrix inversion operation at every timestep which prevents its application in domains with large numbers of state variables. We propose Relational Gaussian Models to represent and model dynamic systems with large numbers of variables efficiently. Furthermore, we devise an exact lifted Kalman Filtering algorithm which takes only linear time in the number of random variables at every timestep. We prove that our algorithm takes linear time in the number of state variables even when individual observations apply to each variable. To our knowledge, this is the first lifted (linear time) algorithm for filtering with continuous dynamic relational models.

Memory heat map: anomaly detection in real-time embedded systems using memory behavior

In this paper, we introduce a novel mechanism that identifies abnormal system-wide behaviors using the predictable nature of real-time embedded applications. We introduce Memory Heat Map (MHM) to characterize the memory behavior of the operating system. Our machine learning algorithms automatically (a) summarize the information contained in the MHMs and then (b) detect deviations from the normal memory behavior patterns. These methods are implemented on top of a multicore processor architecture to aid in the process of monitoring and detection. The techniques are evaluated using multiple attack scenarios including kernel rootkits and shellcode. To the best of our knowledge, this is the first work that uses aggregated memory behavior for detecting system anomalies especially the concept of memory heat maps.

A spatio-temporal pyramid matching for video retrieval

Highlights? We introduce a content-based video retrieval system for a query video shot. ? The shot boundaries are found using a classifier learnt from a boosting algorithm. ? The similarity of video shots is calculated by spatio-temporal pyramid matching. ? The pyramid-matching kernel includes temporal dimension into the matching schema. ? Experiments using sports and UCF50 shows effectiveness of our method. An efficient video retrieval system is essential to search relevant video contents from a large set of video clips, which typically contain several heterogeneous video clips to match with. In this paper, we introduce a content-based video matching system that finds the most relevant video segments from video database for a given query video clip. Finding relevant video clips is not a trivial task, because objects in a video clip can constantly move over time. To perform this task efficiently, we propose a novel video matching called Spatio-Temporal Pyramid Matching (STPM). Considering features of objects in 2D space and time, STPM recursively divides a video clip into a 3D spatio-temporal pyramidal space and compares the features in different resolutions. In order to improve the retrieval performance, we consider both static and dynamic features of objects. We also provide a sufficient condition in which the matching can get the additional benefit from temporal information. The experimental results show that our STPM performs better than the other video matching methods.

Semi-Local Structure Patterns for Robust Face Detection

In many image processing and computer vision problems, including face detection, local structure patterns such as local binary patterns (LBP) and modified census transform (MCT) have been adopted in widespread applications due to their robustness against illumination changes. However, being reliant on the local differences between neighboring pixels, they are inevitably sensitive to noise. To overcome the problem of noise-vulnerability of the conventional local structure patterns, we propose semi-local structure patterns (SLSP), a novel feature extraction method based on local region-based differences. The SLSP is robust to illumination variations, distortion, and sparse noise because it encodes the relative sizes of the central region with locally neighboring regions into a binary code. The principle of SLSP leads noise-robust expansions of LBP and MCT feature extraction frameworks. In a statistical analysis, we find that the proposed methods transform a substantial amount of random noise patterns in face images into more meaningful uniform patterns. The empirical results on the MIT + CMU dataset and FDDB (face detection dataset and benchmark) show that the proposed semi-local patterns applied to LBP and MCT feature extraction frameworks outperform the conventional LBP and MCT features in AdaBoost-based face detectors, with much higher detection rates.

Use of Machine Learning Methods to Reduce Predictive Error of Groundwater Models

Quantitative analyses of groundwater flow and transport typically rely on a physically-based model, which is inherently subject to error. Errors in model structure, parameter and data lead to both random and systematic error even in the output of a calibrated model. We develop complementary data-driven models (DDMs) to reduce the predictive error of physically-based groundwater models. Two machine learning techniques, the instance-based weighting and support vector regression, are used to build the DDMs. This approach is illustrated using two real-world case studies of the Republican River Compact Administration model and the Spokane Valley-Rathdrum Prairie model. The two groundwater models have different hydrogeologic settings, parameterization, and calibration methods. In the first case study, cluster analysis is introduced for data preprocessing to make the DDMs more robust and computationally efficient. The DDMs reduce the root-mean-square error (RMSE) of the temporal, spatial, and spatiotemporal prediction of piezometric head of the groundwater model by 82%, 60%, and 48%, respectively. In the second case study, the DDMs reduce the RMSE of the temporal prediction of piezometric head of the groundwater model by 77%. It is further demonstrated that the effectiveness of the DDMs depends on the existence and extent of the structure in the error of the physically-based model.

Low complexity sensing for big spatio-temporal data

Many large scale sensor networks produce tremendous data, typically as massive spatio-temporal data streams. We present a Low Complexity Sensing framework that, coupled with novel compressive sensing techniques, enables to reduce computational and communication overheads significantly without much compromising the accuracy of sensor readings. More specifically, our sensing framework randomly samples time-series data in the temporal dimension first, then in the spatial dimension. Under some mild conditions, our sensing framework holds the same theoretical bound of reconstruction error, but is much simpler and easier to implement than existing compressive sensing frameworks. In experiments with real world environmental data sets, we demonstrate that the proposed framework outperforms two existing compressive sensing frameworks designed for spatio-temporal data.