Supporting Metacognition during Exploratory Search with the OrgBox

Abstract

Current search systems provide effective support to users engaged in fact-finding and look-up oriented tasks. However, they provide relatively little support for users engaged in exploratory search tasks that involve cognitive and metacognitive activities such as learning, synthesis, planning, and reflection. We conducted a within-subject user study (N=24) that investigated the effects of a novel knowledge organization tool called the OrgBox, designed to assist users with organizing and synthesizing information, and metacognitive activities. The OrgBox included features to allow users to drag-drop information they found through search into boxes that could be created, labelled, and re-arranged. Study participants completed two exploratory search tasks, one with the OrgBox, and one with the OrgDoc, a baseline tool that included features of a rich-text editor (e.g., formatting, bullets) for taking notes. In this paper, we present results from our study comparing the OrgBox and OrgDoc tools. Specifically, we investigate if there were differences in participants (1) search interactions, (2) saving and organizing behaviors (e.g., amount of information, structure of notes), (3) perceptions of the tasks, tool usability, and quality of their task outputs, and (4) perceptions of how the tools provided support for cognitive and metacognitive activities involved in the task. Our results show that when using the OrgBox tool, participants created more grouping sections in their notes and saved more text. In terms of metacognitive support, participants perceived the OrgBox tool to provide significantly higher levels of support for three types of metacognitive activity (monitoring/tracking, evaluation, and planning) without changing their perceptions of the task difficulty.