Einat Amitay

Web-a-where: geotagging web content

We describe Web-a-Where, a system for associating geography with Web pages. Web-a-Where locates mentions of places and determines the place each name refers to. In addition, it assigns to each page a geographic focus --- a locality that the page discusses as a whole. The tagging process is simple and fast, aimed to be applied to large collections of Web pages and to facilitate a variety of location-based applications and data analyses.Geotagging involves arbitrating two types of ambiguities: geo/non-geo and geo/geo. A geo/non-geo ambiguity occurs when a place name also has a non-geographic meaning, such as a person name (e.g., Berlin) or a common word (Turkey). Geo/geo ambiguity arises when distinct places have the same name, as in London, England vs. London, Ontario.An implementation of the tagger within the framework of the WebFountain data mining system is described, and evaluated on several corpora of real Web pages. Precision of up to 82% on individual geotags is achieved. We also evaluate the relative contribution of various heuristics the tagger employs, and evaluate the focus-finding algorithm using a corpus pretagged with localities, showing that as many as 91% of the foci reported are correct up to the country level.

The connectivity sonar: detecting site functionality by structural patterns

Web sites today serve many different functions, such as corporate sites, search engines, e-stores, and so forth. As sites are created for different purposes, their structure and connectivity characteristics vary. However, this research argues that sites of similar role exhibit similar structural patterns, as the functionality of a site naturally induces a typical hyperlinked structure and typical connectivity patterns to and from the rest of the Web. Thus, the functionality of Web sites is reflected in a set of structural and connectivity-based features that form a typical signature. In this paper, we automatically categorize sites into eight distinct functional classes, and highlight several search-engine related applications that could make immediate use of such technology. We purposely limit our categorization algorithms by tapping connectivity and structural data alone, making no use of any content analysis whatsoever. When applying two classification algorithms to a set of 202 sites of the eight defined functional categories, the algorithms correctly classified between 54.5% and 59% of the sites. On some categories, the precision of the classification exceeded 85%. An additional result of this work indicates that the structural signature can be used to detect spam rings and mirror sites, by clustering sites with almost identical signatures.

Social search and discovery using a unified approach

This research explores new ways to augment the search and discovery of relations between Web 2.0 entities using multiple types and sources of social information. Our goal is to allow the search for all object types such as documents, persons and tags, while retrieving related objects of all types. We implemented a social-search engine using a unified approach, where the search space is expanded to represent heterogeneous information objects that are interrelated by several relation types. Our solution is based on multifaceted search, which provides an efficient update mechanism for relations between objects, as well as efficient search over the heterogeneous data. We describe a social search engine positioned within a large enterprise, applied over social data gathered from several Web 2.0 applications. We conducted a large user study with over 600 people to evaluate the contribution of social data for search. Our results demonstrate the high precision of social search results and confirm the strong relationship of users and tags to the topics retrieved.

Introduction to special issue on query log analysis: Technology and ethics

It has been ten years since the first published analysis of a Web search engine query log. The year was 1998, and the results were based on a sample of the AltaVista search query log which did not include any user or session information but sufficed for understanding query caching performance. Since then, there have been many studies surveying the potential use of query logs for improving search engines’ efficacy and usability. Such studies mined the logs to improve numerous search engine capabilities such as query refinement and expansion, spell checking, ranking, targeted advertising, etc. As search engines have become the principal gateways to the Web, their query logs now capture almost the entire Web experience, tracking what users queried for, what results they chose to follow, and what subsequent queries they submitted after viewing the initial results. The value of the logs for research into anything Web related is beyond doubt, but the sheer amount of information captured in the logs makes protecting the user’s privacy difficult. Two years ago, in August 2006, AOL released a sample of a query log for research purposes. The release included the trace of queries submitted to AOL by 500,000 users over a period of three months and included anonymous user IDs. Unfortunately, the anonymization used by AOL could easily be defeated for some users. This fact caught the attention of the media and surfaced many ethical and practical issues not previously debated outside the Web Information Retrieval community. Privacy advocates called for search engine regulation and for increasing users’ awareness and control over the systematic analysis of query data. The negative publicity raised by the AOL incident resulted in the restriction and even the cessation of research collaborations between industry and academia. In editing this special issue, we wanted to present three considerations that should inform the query log use debate. The first and foremost consideration is the privacy concern raised by the use of query logs: The first article appearing in this issue was written by Alice Cooper from the Center for Democracy and Technology (CDT). Cooper surveys the threats to search engine users posed by keeping and analyzing query logs. She studies the currently available