Luanne Freund

Assigning search tasks designed to elicit exploratory search behaviors

The goal of this paper is to provide guidance to researchers investigating exploratory search behaviors and exploratory search systems. It focuses on the design of search tasks assigned in such studies. Based on a review of past studies, a set of task characteristics associated with exploratory search tasks are identified: exploratory search tasks focus on learning and investigative search goals; they are general (rather than specific), open-ended, and often target multiple items/documents; they involve uncertainty and are motivated by ill-defined or ill-structured problems; they are dynamic and evolve over time; they are multi-faceted and may be procedurally complex; and they are often accompanied by other information or cognitive behaviors, such as sensemaking. Recommendations are provided for the design of search task descriptions that will elicit exploratory search behaviors.

Modeling task-genre relationships for IR in the workplace

Context influences the search process, but to date research has not definitively identified which aspects of context are the most influential for information retrieval, and thus are worthy of integration in todays retrieval systems. In this research, we isolated for examination two aspects of context: task and document genre and examined the relationship between them within a software engineering work domain. In this domain, the nature of the task has an impact on decisions of relevance and usefulness, and the document collection contains a distinctive set of genre. Our data set was a document repository created and used by our target population. The document surrogates were meta-tagged by purpose and document type. Correspondence analysis of this categorical data identified some specific relationships between genres and tasks, as well as four broad dimensions of variability underlying these relationships. These results have the potential to inform the design of a contextual retrieval system by refining search results for this domain.

Modeling the Information Behaviour of Software Engineers Using a Work-Task Framework

Faced with the rapid proliferation of digital information resources within organizations, employees need targeted search systems in order to be effective in their jobs. The goal of this study is to develop a model of information behaviour within a work-task framework that can be used to inform the design of a workplace information search system. In a two-phase process, we conducted a series of studies using multiple methods to identify workplace characteristics and to understand how they influence the needs, search strategies, and information sources used by software engineers working as services consultants in a large high-tech company.

Task Effects on Interactive Search: The Query Factor

The purpose of this research is to examine how search differs according to selected task variables. Three types of task information goals and two types of task structures were explored. This mixed within- and between-subjects designed study had 96 participants complete three of 12 tasks in a laboratory setting using a specialized search system based on Lucene. Using a combination metrics (user perception collected by questionnaires, transaction log data, and characteristics of relevant documents), we assessed the effect of goals and structure on search as demonstrated through queries and their use in interactive searching.

Information interaction in 140 characters or less: genres on twitter

In this paper, we describe a genre analysis of Twitter updates, commonly called tweets. The aim was to understand and characterize the communication supported by Twitter in a structured manner enabled by the genre concept. We analyzed six facets of Twitter genres: who, what, where, when, why, and how, and identified a set of five common Twitter genres.

WiIRE: the web interactive information retrieval experimentation system prototype

We introduce WiIRE, a prototype system for conducting: interactive information retrieval (IIR) experiments via the Internet. We conceived WiIRE to increase validity while streamlining procedures and adding efficiencies to the conduct of IIR experiments. The system incorporates password-controlled access, online questionnaires, study instructions and tutorials, conditional interface assignment, and conditional query assignment as well as provision for data collection. As an initial evaluation, we used WiIRE inhouse to conduct a Web-based IIR experiment using an external search engine with customized search interfaces and the TREC 11 Interactive Track search queries. Our evaluation of the prototype indicated significant cost efficiencies in the conduct of IIR studies, and additionally had some novel findings about the human perspective: about half participants would have preferred some personal contact with the researcher, and participants spent a significantly decreasing amount of time on tasks over the course of a session.

Untangling search task complexity and difficulty in the context of interactive information retrieval studies

Purpose – One core element of interactive information retrieval (IIR) experiments is the assignment of search tasks. The purpose of this paper is to provide an analytical review of current practice in developing those search tasks to test, observe or control task complexity and difficulty. Design/methodology/approach – Over 100 prior studies of IIR were examined in terms of how each defined task complexity and/or difficulty (or related concepts) and subsequently interpreted those concepts in the development of the assigned search tasks. Findings – Search task complexity is found to include three dimensions: multiplicity of subtasks or steps, multiplicity of facets, and indeterminability. Search task difficulty is based on an interaction between the search task and the attributes of the searcher or the attributes of the search situation. The paper highlights the anomalies in our use of these two concepts, concluding with suggestions for future methodological research related to search task complexity and d...

Searching for relevance in the relevance of search

Discussion of relevance has permeated the information science literature for the past 50+ years, and yet we are no closer to resolution of the matter. In this research we developed a set of measures to operationalize the dimensions underpinning Saracevics manifestations of relevance. We used an existing data set collected from 48 participants who used a web search engine to complete four search tasks that represent four subject domains. From this study which had assessed multiple aspects of the search process – from cognitive to behavioural – we derived a set of measures for cognitive, motivational, situational, topical and system relevances. Using regression analysis, we demonstrate how the measures partially predict search success, and additionally use factor analysis to identify the underlying constructs of relevance. The results show that Saracevics five manifestations may be merged into three types that represent the user, system and the task.

Contextualizing the information-seeking behavior of software engineers

Information seeking in the workplace can vary substantially from one search to the next due to changes in the context of the search. Modeling these dynamic contextual effects is an important challenge facing the research community because it has the potential to lead to more responsive search systems. With this motivation, a study of software engineers was conducted to understand the role that contextual factors play in shaping their information‐seeking behavior. Research was conducted in the field in a large technology company and comprised six unstructured interviews, a focus group, and 13 in‐depth, semistructured interviews. Qualitative analysis revealed a set of contextual factors and related information behaviors. Results are formalized in the contextual model of source selection, the main contributions of which are the identification of two types of conditioning variables (requirements and constraints) that mediate between the contextual factors and source‐selection decisions, and the articulation of dominant source‐selection patterns. The study has implications for the design of context‐sensitive search systems in this domain and may inform contextual approaches to information seeking in other professional domains.

Towards genre classification for IR in the workplace

Use of document genre in information retrieval systems has the potential to improve the task-appropriateness of results. However, genre classification remains a challenging problem. We describe a case study of genre classification in a software engineering workplace domain, which includes the development of a genre taxonomy and experiments in automatic genre classification using supervised machine learning. We present results based on evaluation using real-life enterprise data from this work domain.