Laszlo Grunfeld

Improving weak ad-hoc retrieval by web assistance and data fusion

Users experience frustration when their reasonable queries retrieve no relevant documents. We call these weak queries and retrievals. Improving their effectiveness is an important issue in ad-hoc retrieval and will be most rewarding for these users. We offer an explanation (with experimental support) why data fusion of sufficiently different retrieval lists can improve weak query results. This approach requires sufficiently different retrieval lists for an ad-hoc query. We propose various ways of selecting salient terms from longer queries to probe the web, and define alternate queries from web results. Target retrievals by the original and alternate queries are combined. When compared with normal ad-hoc retrieval, web assistance and data fusion can improve weak query effectiveness by over 100%. Another benefit of this approach is that other queries also improve along with weak ones, unlike pseudo-relevance feedback which works mostly for non-weak queries.

Employing web mining and data fusion to improve weak ad hoc retrieval

When a user issues a reasonable query to a retrieval system and obtains no relevant documents, he or she is bound to feel frustrated. We call these weak queries and retrievals. Improving their effectiveness is an important issue for ad hoc retrieval and would be most rewarding for these users. We explain why data fusion of sufficiently dissimilar retrieval lists can improve weak query results and confirm this with experiments using short and medium size queries. To realize sufficiently dissimilar retrieval lists, we propose composing alternate queries through web search and mining, employ them for target retrieval, and combine with the original query retrieval list. Methods of forming web probes from longer queries, including salient term selection and query text window rotation, are investigated. When compared with normal ad hoc retrieval, web assistance and data fusion can more than double the original weak query effectiveness. Other queries can also improve along with weak ones.