Yuan-Chi Chang

The onion technique: indexing for linear optimization queries

This paper describes the Onion technique, a special indexing structure for linear optimization queries. Linear optimization queries ask for top-N records subject to the maximization or minimization of linearly weighted sum of record attribute values. Such query appears in many applications employing linear models and is an effective way to summarize representative cases, such as the top-50 ranked colleges. The Onion indexing is based on a geometric property of convex hull, which guarantees that the optimal value can always be found at one or more of its vertices. The Onion indexing makes use of this property to construct convex hulls in layers with outer layers enclosing inner layers geometrically. A data record is indexed by its layer number or equivalently its depth in the layered convex hull. Queries with linear weightings issued at run time are evaluated from the outmost layer inwards. We show experimentally that the Onion indexing achieves orders of magnitude speedup against sequential linear scan when N is small compared to the cardinality of the set. The Onion technique also enables progressive retrieval, which processes and returns ranked results in a progressive manner. Furthermore, the proposed indexing can be extended into a hierarchical organization of data to accommodate both global and local queries.

A low-power, lightweight unit to provide ubiquitous information access application and network support for InfoPad

Some of the most important trends in computer systems are the emerging use of multimedia Internet services, the popularity of portable computing, and the development of wireless data communications. The primary goal of the InfoPad project is to combine these trends to create a system that provides ubiquitous information access. The system is built around a low-power, lightweight wireless multimedia terminal that operates in indoor environments and supports a high density of users. The InfoPad system uses a number of innovative techniques to provide the high-bandwidth connectivity, portability, and user interface needed for this environment. The article describes the design, implementation, and evaluation of the software network and application services that support the InfoPad terminal. Special applications, type servers, and recognizers are developed for the InfoPad system. This software is designed to take advantage of the multimedia capabilities of the portable terminal and the additional computational resources available on the servers. The InfoNet system provides low-latency, high bandwidth connectivity between the computation and the portable terminal. It also provides the routing and handoff support that allows users to roam freely. The performance measurements of the system show that this design is a viable alternative, especially in the indoor environment.

Searching dynamically bundled goods with pairwise relations

Economics research has long recognized that bundling enables savings in production and transaction costs, promotes complementary among the bundle components and sorts consumers according to their valuations. Sellers employ market analysis and intelligence to extract the most surplus. In the age of electronic commerce with low product information access cost, buyers can take advantage of the benefits of bundling by performing dynamic composition of goods from multiple companies offering heterogeneous products and services. These goods, with the proper mix of sources and quantity, may offer additional discounts and benefits, which would not have risen should purchase decisions were made independently. A prominent example is packaged travel, which often involves air, hotel and car rentals. An optimal travel package search not only takes advantage of the lowest available prices of air, hotel and car rental individually but also exploits various discounts through business partnerships between service providers.Todays database infrastructure to support the search of dynamically bundled goods, however, is insufficient. The complex search operations involving cross join of many product categories with hundreds or thousands of offerings can be formulated as SQL queries. But executing these queries in a traditional database is inefficient. This paper proposes an I/O conscious, dynamic programming based algorithm for bundle search. The proposed algorithm finds the top-K combinations of goods abstracted by a linear relationship graph. Experimental results indicate that the proposed algorithm achieves more than two orders of magnitude speedup over cross join, and it is more than an order of-magnitude faster than the simple dynamic programming solution. The performance gap further widens as the number of product categories and the number of offerings within each category increase. This paper characterizes the computational and I/O complexity of the proposed algorithm and suggests extensions to search bundles with more complex relationships.

Distributed application service for Internet information portal

As Internet information portals become prevalent for both Internet and Intranet, most existing Internet Application Server architectures are not scalable to support the large amount of personalization, customization and content adaptation required. We propose a framework to capture the information and content dissemination process. Furthermore, we propose a methodology to map this process to a distributed application server environment. By fully exploiting the intersections of user preference at multiple content processing stages, this new framework enables high hit ratio on processing, storage, and transmission of content and thus scales well to support a large number of clients.

Improving network video quality with delay cognizant video coding

Delay critical network video like videoconferencing requires bandwidth reservations higher than the long-term average rate in order to accommodate its bursty traffic output. While prior research contributed to the estimation of the required bandwidth, which is often known as the effective bandwidth, few if any of the previous work attempted to make use of the rate gap (residual bandwidth) between the average and effective rates. In this paper, we presented an application of delay cognizant video coding (DCVC) to improve the quality of network video by using this residual bandwidth. DCVC generates two compressed video flows with differential delay requirements, one delay-critical flow at the average rate to establish an initial presentation and the second delay-relaxed flow at the residual rate to improve its quality. At no additional cost to end users, significant quality improvement can be achieved and it is demonstrated through both objective and subjective measures of a simulated network video connection.

Event detection in sensor networks for modern oil fields

We report the experience of implementing event detection analytics to monitor and forewarn oil production failures in modern, digitized oil fields. Modern oil fields are equipped with thousands of sensors and gauges to measure various physical and chemical characteristics of oil and gas from underground reservoirs to distribution systems. Data from these massive sensor networks weave a picture depicting the state of oil production and potentially hinting at troubles ahead. Continuous streams of sensor readings can be tapped and fed into analytical algorithms in real time to estimate the likelihood of failure events and generate alerts for possible engineering actions. However, the large amount of main memory required to maintain algorithmic states on cumulative stream data poses challenges to todays web-centric, short-message oriented IT infrastructure. Familiar techniques such as data aggregation, selective sampling and window truncating cannot be applied to some sophisticated algorithms. The paper details our end-to-end solution, points out mismatches with the prevalent transactional web model and suggests new research directions.

Solarspire: querying temporal solar imagery by content

In this paper, we describe a novel content-based retrieval application which permits astrophysicists to search large image sequence archives for solar phenomenon, such as solar flares, based on the spatio-temporal behavior of the solar phenomenon. Specifically, images are preprocessed to identify bright and dark spots based on their relative intensity with respect to their neighboring regions. Temporally persistent objects are then extracted from the collection of spots, and their spatio-temporal behavior represented as intensity and size time series. Users define a query in terms of a model of spatio-temporal behaviors through a Web-based interface. The stored intensity and size time series are searched, and series segments that match the specified specified spatio-temporal behavior are returned. The benchmark results based on 2500 satellite images show that the proposed methodology demonstrated better than 85% accuracy on a solar phenomenon previously identified by astrophysicists.

Supporting Efficient Parametric Search of E-Commerce Data: A Loosely-Coupled Solution

Electronic commerce is emerging as a major application area for database systems. A large number of e-commerce sites provide electronic product catalogs that allow users to search products of interest.Due to the constant evolution and the high sparsity of e-commerce data, most commercial e-commerce systems use the so-called vertical schema for data storage. However, query processing for data stored using vertical schema is extremely slow because current RDBMS, especially its cost-based query optimizer, is designed to deal with traditional horizontal schema efficiently.Most e-commerce systems would like to offer advanced parametric search capabilities to their users. However, most searches are expected to be on-line which means that the query execution should be very fast. RDBMSs require new capabilities and enhancements before they can satisfy the search performance criteria against vertical schema. The tightly-coupled enhancements and additions to a DBMS require considerable amount of work and may take a long time to be accomplished. In this paper, we describe an alternative approach called SAL, a Search Assistant Layer that can be implemented outside a database engine to accommodate the urgent need for efficient parametric search on e-commerce data. Our experimental results show that dramatic performance improvement is provided by SAL for search queries.

System G Data Store: Big, Rich Graph Data Analytics in the Cloud

Big, rich graph data is increasingly captured through the interactions among people (email, messaging, social media), objects (location/map, server/network, product/catalog) and their relations. Graph data analytics, however, poses several intrinsic challenges that are ill fitted to the popular Map Reduce programming model. This paper presents System G, a graph data management system that supports rich graph data, accepts online updates, complies with Hadoop, and runs efficiently by minimizing redundant data shuffling. These desirable capabilities are built on top of Apache HBase for scalability, updatability and compatibility. This paper introduces several exemplary target graph queries and global feature algorithms implemented using the newly available HBase Coprocessors. These graph algorithmic coprocessors execute on the server side directly on graph data stored locally and only communicates with remote servers for the dynamic algorithmic state, which is typically a small fraction of the raw data. Performance evaluation on real-world rich graph datasets demonstrated significant improvement over traditional Hadoop implementation, as prior works observed in their no-graph-shuffling solutions. Our work stands out at achieving the same or better performance without introducing incompatibility or scalability limitations.

Framework for efficient processing of content-based fuzzy Cartesian queries

It has become increasingly important for multimedia databases to provide capabilities for content-based retrieval of multi-modal data at multiple abstraction levels for various decision support applications. These decision support applications commonly require the evaluation of fuzzy spatial or temporal Cartesian product of objects that have been retrieved based on their similarity to the target object in terms of color, shape or texture features.