Ralf Bierig

Search behaviors in different task types

Personalization of information retrieval tailors search towards individual users to meet their particular information needs by taking into account information about users and their contexts, often through implicit sources of evidence such as user behaviors. Task types have been shown to influence search behaviors including usefulness judgments. This paper reports on an investigation of user behaviors associated with different task types. Twenty-two undergraduate journalism students participated in a controlled lab experiment, each searching on four tasks which varied on four dimensions: complexity, task product, task goal and task level. Results indicate regular differences associated with different task characteristics in several search behaviors, including task completion time, decision time (the time taken to decide whether a document is useful or not), and eye fixations, etc. We suggest these behaviors can be used as implicit indicators of the users task type.

Task and user effects on reading patterns in information search

We report on an investigation into people’s behaviors on information search tasks, specifically the relation between eye movement patterns and task characteristics. We conducted two independent user studies (n = 32 and n = 40), one with journalism tasks and the other with genomics tasks. The tasks were constructed to represent information needs of these two different users groups and to vary in several dimensions according to a task classification scheme. For each participant we classified eye gaze data to construct models of their reading patterns. The reading models were analyzed with respect to the effect of task types and Web page types on reading eye movement patterns. We report on relationships between tasks and individual reading behaviors at the task and page level. Specifically we show that transitions between scanning and reading behavior in eye movement patterns and the amount of text processed may be an implicit indicator of the current task type facets. This may be useful in building user and task models that can be useful in personalization of information systems and so address design demands driven by increasingly complex user actions with information systems. One of the contributions of this research is a new methodology to model information search behavior and investigate information acquisition and cognitive processing in interactive information tasks.

Time, location and interest: an empirical and user-centred study

The importance of context in meeting user information needs has gained increasing interest. When developing interactive information retrieval systems, we do need to consider how contextual information might be used to improve information retrieval. In this paper, we present a user-centred experiment that focuses on three potential context attributes. These are time, location, and users interest. The experiment involved tasks using a scenario that would be suitable for mobile situations - one very promising area for the application of context information that can help to deliver personalised services. The scenario involves situations with local events such as jazz concerts and includes the use of a simplified map to help visualise locations. The effect of the three attributes and the interactions between them are analysed and discussed. The effects in most cases were considerable and data analysis showed statistically significant effects. The study shows that time, location, and interest matter to users in mobile situations. There appears to be a priority emerging in the relative importance of these attributes for the mobile user. Also, the results show high order interaction effects between the attributes.

Linking search tasks with low-level eye movement patterns

Motivation -- On-the-task detection of the task type and task attributes can benefit personalization and adaptation of information systems. Research approach -- A web-based information search experiment was conducted with 32 participants using a multi-stream logging system. The realistic tasks were related directly to the backgrounds of the participants and were of distinct task types. Findings/Design -- We report on a relationship between task and individual reading behaviour. Specifically we show that transitions between scanning and reading behaviour in eye movement patterns are an implicit indicator of the current task. Research limitations/Implications -- This work suggests it is plausible to infer the type of information task from eye movement patterns. One limitation is a lack of knowledge about the general reading model differences across different types of tasks in the population. Although this is an experimental study we argue it can be generalized to real world text-oriented information search tasks. Originality/Value -- This research presents a new methodology to model user information search task behaviour. It suggests promise for detection of information task type based on patterns of eye movements. Take away message -- With increasingly complex computer interaction, knowledge about the type of information task can be valuable for system personalization. Modelling the reading/scanning patterns of eye movements can allow inference about the task type and task attributes.

Contextual factors affecting the utility of surrogates within exploratory search

In this paper we investigate how information surrogates might be useful in exploratory search and what information it is useful for a surrogate to contain. By comparing assessments based on artificially created information surrogates, we investigate the effect of the source of information, the quality of an information source and the date of information upon the assessment process. We also investigate how varying levels of topical knowledge, assessor confidence and prior expectation affect the assessment of information surrogates. We show that both types of contextual information affect how the information surrogates are judged and what actions are performed as a result of the surrogates.

A Combined Approach of Structured and Non-structured IR in Multimodal Domain

We present a generic model for multimodal information retrieval, leveraging different information sources to improve the effectiveness of a retrieval system. The proposed method is able to take into account both explicit and latent semantics present in the data and can be used to answer complex queries, not currently answerable neither by document retrieval systems, nor by semantic web systems. By providing a hybrid approach combining IR and structured search techniques, we prepare a framework applicable to multimodal data collections. To test its effectiveness, we instantiate the model for an image retrieval task.

A Hybrid Approach for Multi-faceted IR in Multimodal Domain

We present a model for multimodal information retrieval, leveraging different information sources to improve the effectiveness of a retrieval system. This method takes into account multifaceted IR in addition to the semantic relations present in data objects, which can be used to answer complex queries, combining similarity and semantic search. By providing a graph data structure and utilizing hybrid search in addition to structured search techniques, we take advantage of relations in data to improve retrieval. We tested the model with ImageCLEF 2011 Wikipedia collection, as a multimodal benchmark data collection, for an image retrieval task.

Are self-assessments reliable indicators of topic knowledge?

Self-assessment of topic/task knowledge is a human metacognitive capacity that impacts information behavior, for example through selection of learning and search strategies. It is often used as a measure in experiments for evaluation of results and those measurements are taken to be generally reliable. We conducted a user study (n=40) to test this by constructing a concept-based topic knowledge representation for each participant and then comparing it with the participant judgment of their topic knowledge elicited with Likert-scale questions. The tasks were in the genomics domain and knowledge representations were constructed from the MeSH thesaurus terms that indexed relevant documents for five topics. The participants rated their familiarity with the topic, the anticipated task difficulty, the amount of learning gained during the task, and made other knowledge-related judgments associated with the task. Although there is considerable variability over individuals, the results provide evidence that these self-assessed topic knowledge measures are correlated in the expected way with the independently-constructed topic knowledge measure. We argue the results provide evidence for the general validity of topic knowledge self-assessment and discuss ways to further explore knowledge self-assessment and its reliability for prediction of individual knowledge levels.

On the use of statistical semantics for metadata-based social image retrieval

We revisit text-based image retrieval for social media, exploring the opportunities offered by statistical semantics. We assess the performance and limitation of several complementary corpus-based semantic text similarity methods in combination with word representations. We compare results with state-of-the-art text search engines. Our deep learning-based semantic retrieval methods show a statistically significant improvement in comparison to a best practice Solr search engine, at the expense of a significant increase in processing time. We provide a solution for reducing the semantic processing time up to 48% compared to the standard approach, while achieving the same performance.

SIGIR 2009 workshop on understanding the user: logging and interpreting user interactions in information search and retrieval

Modern information search systems can benefit greatly from using additional information about the user and the users behavior, and research in this area is active and growing. Feedback data based on direct interaction (e.g., clicks, scrolling, etc.) as well as on user profiles/preferences has been proven valuable for personalizing the search process, e.g., from how queries are understood to how relevance is assessed. New technology has made it inexpensive and easy to collect more feedback data and more different types of data (e.g., gaze, emotional, or biometric data). The workshop “Understanding the User – Logging and interpreting user interactions in information search and retrieval” was held in conjunction with the 32nd Annual International ACM SIGIR Conference. It focused on discussing and identifying most promising research directions with respect to logging, interpreting, integrating, and using feedback data. The workshop aimed at bringing together researchers especially from the domains of IR and human-computer interaction interested in the collection, interpretation, and application of user behavior logging for search. Ultimately, one of the main goals was to arrange a commonly shared collection of user interaction logging tools based on a variety of feedback data sources as well as best practices for their usage.