Byunggu Yu

On Managing Very Large Sensor-Network Data Using Bigtable

Recent advances and innovations in smart sensor technologies, energy storage, data communications, and distributed computing paradigms are enabling technological breakthroughs in very large sensor networks. There is an emerging surge of next-generation sensor-rich computers in consumer mobile devices as well as tailor-made field platforms wirelessly connected to the Internet. Billions of such sensor computers are posing both challenges and opportunities in relation to scalable and reliable management of the peta- and exa-scale time series being generated over time. This paper presents a Cloud-computing approach to this issue based on the two well-known data storage and processing paradigms: Bigtable and MapReduce.

A retrieval technique for high-dimensional data and partially specified queries

While the persistent data of many advanced database applications, such as OLAP and scientific studies, are characterized by very high dimensionality, typical queries posed on these data appeal to a small number of relevant dimensions. Unfortunately, the multi-dimensional access methods designed for high-dimensional data perform rather poorly for these partially specified queries. The retrieval technique proposed in this paper uses a combination of two complementary measures to support efficient processing of partial queries over high-dimensional data. First, an elaborate storage organization, called the inverted space, allows the system administrator to control the size of individual indexes in order to avoid the negative impact of extremely high data dimensionality on the retrieval performance. Second, a new indexing structure, which is designed to support the inverted-space storage organization, enables efficient query processing in projected spaces with moderate dimensionality. This indexing mechanism is a general-purpose point access method that effectively attacks the limitations of KDB-trees in spaces with many dimensions, while preserving the simplicity and relatively good performance of the later structure in low-dimensional spaces. The analytical and experimental results show that the new indexing scheme outperforms two other variants of KDB-trees investigated in the paper for both fully and partially specified queries.

A survey of cloud-based network intrusion detection analysis

As network traffic grows and attacks become more prevalent and complex, we must find creative new ways to enhance intrusion detection systems (IDSes). Recently, researchers have begun to harness both machine learning and cloud computing technology to better identify threats and speed up computation times. This paper explores current research at the intersection of these two fields by examining cloud-based network intrusion detection approaches that utilize machine learning algorithms (MLAs). Specifically, we consider clustering and classification MLAs, their applicability to modern intrusion detection, and feature selection algorithms, in order to underline prominent implementations from recent research. We offer a current overview of this growing body of research, highlighting successes, challenges, and future directions for MLA-usage in cloud-based network intrusion detection approaches.

Implementing KDB-trees to support high-dimensional data

The problem of retrieving large volumes of high dimensional data is an important and timely issue in the area of database management. The guiding idea of the paper is to develop a general-purpose point access method that attacks the limitations of KDB-trees in high-dimensional spaces, while preserving their relatively good performance in low-dimensional situations. The proposed structure, called high-dimensional KDB-tree, eliminates downward propagation of splits associated with the original KDB-tree structure, which results in low storage utilization and rapid deterioration of the retrieval performance. Additional improvements in the storage and retrieval performance are achieved by removing certain redundant information from the interior nodes. Experimental results show that, in high-dimensional spaces, the proposed structure outperforms the original KDB-trees by a significant margin, while incurring no loss of performance in low-dimensional spaces. The structure also outperforms two other variants of KDB-trees investigated in the paper.

KDB/sub KD/-tree: a compact KDB-tree structure for indexing multidimensional data

The problem of storing and retrieving high-dimensional data continues to be an important issue. In this paper, we propose an efficient high-dimensional point access method called the KDB/sub KD/-tree. The KDBKD-tree eliminates redundant information in KDB-trees by changing the representation of the index entries in the interior pages. Experimental evidence shows that the KDB/sub KD/-tree outperforms other recent variants of KDB-trees, such as KDB/sub FD/-trees and KDB/sub HD/-trees.

An integrated framework for managing sensor data uncertainty using cloud computing

In recent years, an increasing number of data-intensive applications deal with continuously changing data objects (CCDOs), such as data streams from sensors and tracking devices. In these applications, the underlying data management system must support new types of spatiotemporal queries that refer to the spatiotemporal trajectories of the CCDOs. In contrast to traditional data objects, CCDOs have continuously changing attributes. Therefore, the spatiotemporal relation between any two CCDOs can change over time. This problem can be more complicated, since the CCDO trajectories are associated with a degree of uncertainty at every point in time. This is due to the fact that databases can only be discretely updated. The paper formally presents a comprehensive framework for managing CCDOs with insights into the spatiotemporal uncertainty problem and presents an original parallel-processing solution for efficiently managing the uncertainty using the map-reduce platform of cloud computing.

Curve-based representation of moving object trajectories

In recent years, many emerging database applications deal with continuously moving data objects - each data object moves continuously and frequently reports its current location, moving direction, and speed to the database server. A database server for these applications keeps track of the trajectories of individual moving objects and processes queries referring to the past or future trajectories. Related techniques view a moving object trajectory as a sequence of connected line segments. However, most natural moving objects, such as airplanes, vessels, and vehicles, draw a smooth trajectory with no angles. This paper presents our curve-based trajectory representation models. The presented results show that the curve-based models provide much more accurate trajectories than the line-based models when we have the same amount of data (same number of reported points). In other words, the curve-based models require a smaller amount of data while providing the same accuracy in trajectory representation.

Vector model in support of versatile georeferenced video search

Increasingly geographic properties are being associated with videos, especially those captured from mobile cameras. The meta data from camera-attached sensors can be used to model the coverage area of the scene as a spatial object such that videos can be organized, indexed and searched based on their field of views (FOV). The most accurate representation of an FOV is through the geometric shape of a circular sector. However, spatial search and indexing methods are traditionally optimized for rectilinear shapes because of their simplicity. Established methods often use an approximation shape, such as a minimum bounding rectangle (MBR), to efficiently filter a large archive for possibly matching candidates. A second, refinement step is then applied to perform the time-consuming, precise matching function. MBR estimation has been successful for general spatial overlap queries, however it provides limited flexibility for georeferenced video search. In this study we propose a novel vector-based model for FOV estimation which provides a more versatile basis for georeferenced video search while providing competitive performance for the filter step. We demonstrate how the vector model can provide a unified method to perform traditional overlap queries while also enabling searches that, for example, concentrate on the vicinity of the cameras position or harness its view direction. To the best of our knowledge no comparable technique exists today.

Simple QSF-trees: an efficient and scalable spatial access method

The development of high-performance spatial access methods that can support complex operations of large spatial databases continues to attract considerable attention. This paper introduces <italic>QSF</italic>-trees, an efficient and scalable structure for indexing spatial objects, which has some important advantages over <italic>R</italic><supscrpt>*</supscrpt>-trees. <italic>QSF</italic>-trees eliminate overlapping of index regions without forcing object clipping or sacrificing the selectivity of spatial operations. The method exploits the semantics of topological relations between spatial objects to further reduce the number of index nodes visited during the search. A series of experiments involving randomly-generated spatial objects was conducted to compare the structure with two variations of <italic>R</italic><supscrpt>*</supscrpt>-trees. The experiments show <italic>QSF</italic>-trees to be more efficient and more scalable to the increase in the data-set size, the size of spatial objects, and the number of dimensions of the spatial universe.

A Bigtable/MapReduce-Based Cloud Infrastructure for Effectively and Efficiently Managing Large-Scale Sensor Networks

This paper proposes a novel approach for effectively and efficiently managing large-scale sensor networks defining a Cloud infrastructure that makes use of Bigtable at the data layer and MapReduce at the processing layer. We provide principles and architecture of our proposed infrastructure along with its experimental evaluation on a real-life computational platform. Experiments clearly confirm the effectiveness and the efficiency of the proposed research.