Chavdar Botev

XRANK: ranked keyword search over XML documents

We consider the problem of efficiently producing ranked results for keyword search queries over hyperlinked XML documents. Evaluating keyword search queries over hierarchical XML documents, as opposed to (conceptually) flat HTML documents, introduces many new challenges. First, XML keyword search queries do not always return entire documents, but can return deeply nested XML elements that contain the desired keywords. Second, the nested structure of XML implies that the notion of ranking is no longer at the granularity of a document, but at the granularity of an XML element. Finally, the notion of keyword proximity is more complex in the hierarchical XML data model. In this paper, we present the XRANK system that is designed to handle these novel features of XML keyword search. Our experimental results show that XRANK offers both space and performance benefits when compared with existing approaches. An interesting feature of XRANK is that it naturally generalizes a hyperlink based HTML search engine such as Google. XRANK can thus be used to query a mix of HTML and XML documents.

Texquery: a full-text search extension to xquery

One of the key benefits of XML is its ability to represent a mix of structured and unstructured (text) data. Although current XML query languages such as XPath and XQuery can express rich queries over structured data, they can only express very rudimentary queries over text data. We thus propose TeXQuery, which is a powerful full-text search extension to XQuery. TeXQuery provides a rich set of fully composable full-text search primitives,such as Boolean connectives, phrase matching, proximity distance, stemming and thesauri. TeXQuery also enables users to seamlessly query over both structured and text data by embedding TeXQuery primitives in XQuery, and vice versa. Finally, TeXQuery supports a flexible scoring construct that can be used toscore query results based on full-text predicates. TeXQuery is the precursor ofthe full-text language extensions to XPath 2.0 and XQuery 1.0 currently being developed by the W3C.

WYSIWYG development of data driven web applications

An emerging trend in Social Networking sites and Web portals is the opening up of their APIs to external application developers. For example, the Facebook Platform, Google Gadgets and Yahoo! Widgets allow developers to design their own applications, which can then can be integrated with the platform and shared with other users. However, current APIs are targeted towards developers with programming expertise and database knowledge; they are not accessible to a large class of users who do not have a programming/database background, but would nevertheless like to create new applications. To address this need, we have developed the AppForge system, which provides a WYSIWYG application development platform. Users can graphically specify the components of webpages inside a Web browser, and the corresponding database schema and application logic will be automatically generated on the fly by the system. The WYSIWYG interface gives instantaneous feedback on what users have created and allows them to run, test and continuously refine their applications. AppForge has been used to create prototype versions of a variety of applications such as an event planning system, a recruiting system, an item trading system and an online course management system. We have also conducted a small and preliminary user study to identify and fix some of the usability aspects of AppForge.

Supporting workflow in a course management system

CMS is a secure and scalable web-based course management system developed by the Cornell University Computer Science Department. The system was designed to simplify, streamline, and automate many aspects of the workflow associated with running a large course, such as course creation, importing students, management of student workgroups, online submission of assignments, assignment of graders, grading, handling regrade requests, and preparation of final grades. In contrast, other course management systems of which we are aware provide only specialized solutions for specific components, such as grading. CMS is increasingly widely used for course management at Cornell University. In this paper we articulate the principles we followed in designing the system and describe the features that users found most useful.

XQuery full-text extensions explained

There has been recent interest in developing XML query languages, such as XPath and XQuery, to tap the vast amount of information represented and stored in Extensible Markup Language (XML). These query languages, however, have focused mainly on querying the structure of XML documents and provide only rudimentary support for querying text content. To fill this void, XQuery Full-Text has been developed as a full-text extension to XQuery (and also XPath, which is a subset of XQuery). Consequently, XQuery Full-Text can be used to seamlessly query over both the structure and the text content of XML documents. This paper explains the design principles behind XQuery Full-Text, describes its evolution, and illustrates its core features with examples. It is intended as a reference that is shorter and more accessible than the current World Wide Web Consortium working draft.

Expressiveness and performance of full-text search languages

We study the expressiveness and performance of full-text search languages. Our motivation is to provide a formal basis for comparing full-text search languages and to develop a model for full-text search that can be tightly integrated with structured search. We design a model based on the positions of tokens (words) in the input text, and develop a full-text calculus (FTC) and a full-text algebra (FTA) with equivalent expressive power; this suggests a notion of completeness for full-text search languages. We show that existing full-text languages are incomplete and identify a practical subset of the FTC and FTA that is more powerful than existing languages, but which can still be evaluated efficiently.

A TeXQuery-based XML full-text search engine

We demonstrate an XML full-text search engine that implements the TeXQuery language. TeXQuery is a powerful full-text search extension to XQuery that provides a rich set of fully composable full-text primitives, such as phrase matching, proximity distance, stemming and thesauri. TeXQuery enables users to seamlessly query over both structure data and text, by embedding full-text primitives in XQuery and vice versa. TeXQuery also supports a flexible scoring construct that scores query results based on full-text predicates and permits top-k queries. TeXQuery is the precursor of the full-text language extension to XPath 2.0 and XQuery 1.0 currently being developed by W3C.

Quark: an efficient XQuery full-text implementation

The XQuery 1.0 and XPath 2.0 Full-text (XQFT) language has been developed by the W3C to extend XQuery and XPath with full-text search capabilities. XQFT allows users to specify a mix of structured and complex full-text predicates, and also allows users to score/rank such queries. The power and flexibility of XQFT gives rise to two interesting questions. First, is it possible to efficiently integrate a full-function XML query language with sophisticated full-text search? Second, is it possible to score and rank arbitrary XQuery and XQFT queries? In this demonstration, we present evidence that it is indeed possible to achieve the above goals. We demonstrate the Quark open-source data management system and show how we can seamlessly and efficiently integrate structured and unstructured search over XML data. In particular, we demonstrate (a) techniques for efficiently evaluating keyword search over virtual XML views, and (b) a framework for scoring both structured and full-text predicates.

Static score bucketing in inverted indexes

Maintaining strict static score order of inverted lists is a heuristic used by search engines to improve the quality of query results when the entire inverted lists cannot be processed. This heuristic, however, increases the cost of index generation and requires complex index build algorithms. In this paper, we study a new index organization based on static score bucketing. We show that this new technique significantly improves in index build performance while having minimal impact on the quality of search results.