Silvia Quarteroni

An Introduction to Information Retrieval

Information retrieval is a discipline that deals with the representation, storage, organization, and access to information items. The goal of information retrieval is to obtain information that might be useful or relevant to the user: library card cabinets are a “traditional” information retrieval system, and, in some sense, even searching for a visiting card in your pocket to find out a colleague’s contact details might be considered as an information retrieval task. In this chapter we introduce information retrieval as a scientific discipline, providing a formal characterization centered on the notion of relevance. We touch on some of its challenges and classic applications and then dedicate a section to its main evaluation criteria: precision and recall.

Web Information Retrieval

With the proliferation of huge amounts of (heterogeneous) data on the Web, the importance of information retrieval (IR) has grown considerably over the last few years. Big players in the computer industry, such as Google, Microsoft and Yahoo!, are the primary contributors of technology for fast access to Web-based information; and searching capabilities are now integrated into most information systems, ranging from business management software and customer relationship systems to social networks and mobile phone applications. Ceri and his co-authors aim at taking their readers from the foundations of modern information retrieval to the most advanced challenges of Web IR. To this end, their book is divided into three parts. The first part addresses the principles of IR and provides a systematic and compact description of basic information retrieval techniques (including binary, vector space and probabilistic models as well as natural language search processing) before focusing on its application to the Web. Part two addresses the foundational aspects of Web IR by discussing the general architecture of search engines (with a focus on the crawling and indexing processes), describing link analysis methods (specifically Page Rank and HITS), addressing recommendation and diversification, and finally presenting advertising in search (the main source of revenues for search engines). The third and final part describes advanced aspects of Web search, each chapter providing a self-contained, up-to-date survey on current Web research directions. Topics in this part include meta-search and multi-domain search, semantic search, search in the context of multimedia data, and crowd search. The book is ideally suited to courses on information retrieval, as it covers all Web-independent foundational aspects. Its presentation is self-contained and does not require prior background knowledge. It can also be used in the context of classic courses on data management, allowing the instructor to cover both structured and unstructured data in various formats. Its classroom use is facilitated by a set of slides, which can be downloaded from www.search-computing.org.

A bottom-up, knowledge-aware approach to integrating and querying web data services

As a wealth of data services is becoming available on the Web, building and querying Web applications that effectively integrate their content is increasingly important. However, schema integration and ontology matching with the aim of registering data services often requires a knowledge-intensive, tedious, and error-prone manual process. We tackle this issue by presenting a bottom-up, semi-automatic service registration process that refers to an external knowledge base and uses simple text processing techniques in order to minimize and possibly avoid the contribution of domain experts in the annotation of data services. The first by-product of this process is a representation of the domain of data services as an entity-relationship diagram, whose entities are named after concepts of the external knowledge base matching service terminology rather than being manually created to accommodate an application-specific ontology. Second, a three-layer annotation of service semantics (service interfaces, access patterns, service marts) describing how services “play” with such domain elements is also automatically constructed at registration time. When evaluated against heterogeneous existing data services and with a synthetic service dataset constructed using Google Fusion Tables, the approach yields good results in terms of data representation accuracy. We subsequently demonstrate that natural language processing methods can be used to decompose and match simple queries to the data services represented in three layers according to the preceding methodology with satisfactory results. We show how semantic annotations are used at query time to convert the users request into an executable logical query. Globally, our findings show that the proposed registration method is effective in creating a uniform semantic representation of data services, suitable for building Web applications and answering search queries.

A Framework for Integrating, Exploring, and Searching Location-Based Web Data

This article presents the adaptation of a general search computing framework for exploratory search over Web data as suggested by the specificity of location-based data services. The result is a conceptual model of geographic entities, the spatial functions operating on them, and a special-purpose exploratory interface that lets users search combinations of georeferenced objects directly on a map. Such modifications help the general framework provide ranked extraction of relevant objects and their combinations, custom ranking functions, and cost-based access to location-based services.

Semantic resource framework

The Semantic Resource Framework (SRF) is a multi-level description of the data sources for search computing applications. It responds to the need of having a structured representation of search services, amenable to service exploration, selection, and invocation. The SRF aims at extending the Service Mart model used so far in search computing to overcome some of its limitations. The main new features include external attributes, which represent the input to be provided by users for accessing objects; selector attributes, describing the possibility to map the same access pattern to different services based on some condition; key attributes for objects; and a generalized notion of nearness between objects. The high-level view presented by SRF is a very simple Entity-Relationship model with objects and binary connections, that can be used for very different query tasks, ranging from custom search applications (i.e. predefined queries) to exploratory search (i.e., exploration of its objects and connections) to natural language interfaces (i.e., query dialogues). Such high-level view should be considered as an initial step in the enrichment of the service repository with additional semantic capabilities.

The Information Retrieval Process

What does an information retrieval system look like from a bird’s eye perspective? How can a set of documents be processed by a system to make sense out of their content and find answers to user queries? In this chapter, we will start answering these questions by providing an overview of the information retrieval process. As the search for text is the most widespread information retrieval application, we devote particular emphasis to textual retrieval. The fundamental phases of document processing are illustrated along with the principles and data structures supporting indexing.

Classification and Clustering

Information overload can be addressed through machine learning techniques that organize and categorize large amounts of data. The two main techniques are classification and clustering. Classification is a supervised technique that assigns a class to each data item by performing an initial training phase over a set of human annotated data and then a subsequent phase which applies the classification to the remaining elements. Clustering is an unsupervised technique that does not assume a priori knowledge: data are grouped into categories on the basis of some measure of inherent similarity between instances, in such a way that objects in one cluster are very similar (compactness property) and objects in different clusters are different (separateness property). Similarity must be calculated in different ways based on the data types it involves, namely categorical, numeric, textual, or mixed data. After the clusters are created, a labeling phase takes care of annotating each cluster with the most descriptive label.

Meta-search and Multi-domain Search

While search engine technology based on crawling and indexing, presented in Part II, dominates the market, a niche is open for search systems based on data integration technology. These systems either rely on other search engines as sources of information or directly access specialized data sources that are focused on given domains. Interest in such systems is growing with the increase of Web applications which offer simple query interfaces to domain-specific data sources. This chapter overviews the theory of rank-driven data integration and top-k query processing, and then focuses on meta-search and multi-domain search.

Natural Language Processing for Search

Unstructured data, i.e., data that has not been created for computer usage, make up about 80 % of the entire amount of digital documents. Most of the time, unstructured data are textual documents written in natural language: clearly, this kind of data is a powerful information source that needs to be handled well. Access to unstructured data may be greatly improved with respect to traditional information retrieval methods by using deep language understanding methods. In this chapter, we provide a brief overview of the relationship between natural language processing and search applications. We describe some machine learning methods that are used for formalizing natural language problems in probabilistic terms. We then discuss the main challenges behind automatic text processing, focusing on question answering as a representative example of the application of various deep text processing techniques.

Search Process and Interfaces

Traditional information retrieval is based on a simple paradigm: driven by an information need, users seek information by composing a query, using a search system; the system, in turn, associates relevant documents to the query and returns them to the users. However, search is more a means than an end, and the needs behind an information seeking process are typically diverse and articulated. Thus, interaction with search systems typically involves several steps of refinement and exploration. Visual interfaces are designed to effectively support this iterative process, help users to understand and express their information needs, and collect the results when the process ends. This chapter provides insight into the main theoretical models used to describe the information seeking process, and offers an overview of the user interface components used by modern search engines.