Saraschandra Karanam

CoLiDeS+ Pic: a cognitive model of web-navigation based on semantic information from pictures

Comprehension-based linked model of deliberate search (CoLiDeS) + Pic is a cognitive model of web-navigation that takes into account the semantic information from graphical elements present on a web-page to compute the information scent value of the hyperlinks. The model is based on CoLiDeS, which has a two-phase processing cycle: (a) attention phase, which first parses the web-page and focuses attention on the region of the web-page that is semantically most similar to the goal, and (b) action-selection phase, which evaluates the available actions in the focused region and selects a particular action such as clicking a link. The graphical elements are important both for attracting attention to a region of the web-page and for communicating semantic meaning that may alter or enhance the meaning of the hyperlink labels. In the first part of this article, we give a theoretical explanation of the CoLiDeS + Pic model and describe the methodology followed to implement it. In the second part, we run a simulation on a mock-up website and evaluate the effect of pictures on information scent of hyperlinks by means of the CoLiDeS + Pic model on basis of the simulation results. It was found that CoLiDeS + Pic with highly relevant pictures increases the values of information scent of task-relevant hyperlinks, and therefore it increases the probability of selecting those hyperlinks compared to CoLiDeS (without pictures) or CoLiDeS + Pic with lowly relevant pictures. These results confirm the importance of including information from pictures into the modelling of web-navigation.

Evaluating CoLiDeS + Pic: the role of relevance of pictures in user navigation behaviour

CoLiDeS + Pic is a cognitive model of web-navigation that incorporates semantic information from pictures intoCoLiDeS. In our earlier research, we have demonstrated that by incorporating semantic information from pictures, CoLiDeS + Pic can predict the hyperlinks on the shortest path more frequently, and also with greater information scent, compared to earlier cognitive models of web-navigation like CoLiDeS that relied only on textualinformation from hyperlinks. In this article, we investigate the following research questions. First, would the increase in information scent have an impact on the actual user navigation behaviour? Second, do users actually follow the navigation path predicted by CoLiDeS + Pic? In other words, would CoLiDeS + Pic predict actual user navigation behaviour more accurately than CoLiDeS? We investigate these questions by varying the relevance of pictures on aweb page and studying the impact of varying relevance on the user navigation patterns. We found that under the highly relevant picture condition, users were more accurate and took less time to finish their tasks. Also, under thehighly relevant picture condition, CoLiDeS + Pic predicts significantly greater number of actual user clicks. There was no significant difference in model predictions between the lowly relevant picture condition and no-picture condition. These results validate the predictions made by CoLiDeS + Pic.

CrowdUtility: know the crowd that works for you

Crowdsourcing platforms aim to leverage the collective intelligence of a largely distributed Internet workforce to solve a wide range of tasks. Crowd workers (unlike in a typical organization), exhibit varying work patterns, expertise, and performance - with little or no control that can be imposed on them. Requesters (e.g. enterprises) also exhibit diverse requirements in terms of the size, complexity and timings of the tasks, as well as SLAs (performance expectations). Clearly, the heterogeneity makes the choice of a platform suited for a given task difficult for the user. This paper highlights this problem and proposes CrowdUtility - a first-of-a-kind statistical machine learning approach, which models the dynamic behavioral characteristics of crowdsourcing platforms and uses them to recommend the best platform for the enterprise task(s). Initial results from real-world experiments suggest that the proposed system provides an attractive solution to this erstwhile unsolved problem

Towards a fully computational model of web-navigation

In this paper, we make the first steps towards developing a fully automatic tool for supporting users for navigation on the web. We developed a prototype that takes a user-goal and a website URL as input and predicts the correct hyperlink to click on each web page starting from the home page, and uses that as support for users. We evaluated our systems usefulness with actual data from real users. It was found that users took significantly less time and less clicks; were significantly less disoriented and more accurate with systemgenerated support; and perceived the support positively. Projected extensions to this system are discussed.

Performance of computational cognitive models of web-navigation on real websites

Computational cognitive models of web-navigation developed so far have largely been tested only on mock-up websites. In this paper, for the first time, we compare and contrast the performance of two models, CoLiDeS and CoLiDeS+, on two real websites from the domains of technology and health, under two conditions of task difficulty, simple and difficult. We found that CoLiDeS+ predicted more hyperlinks on the correct path and had a higher path completion ratio than CoLiDeS. CoLiDeS+ found the target page more often than CoLiDeS, took more steps to reach the target page and was more ‘disoriented’ than CoLiDeS for difficult tasks. Difficult tasks in general for both models had less task success and lower path completion ratio, predicted less hyperlinks on the correct path, visited pages with lower mean LSA and took more steps to complete compared with simple tasks. Overall, inclusion of context from previously visited pages and implementation of backtracking strategies (which are both part of CoLiDeS+) led to better modelling performance. Suggestions to further improve the performance of these computational cognitive models on real websites are discussed.

Modeling and predicting information search behavior

This paper looks at two limitations of cognitive models of web-navigation: first, they do not account for the entire process of information search and second, they do not account for the differences in search behavior caused by aging. To address these limitations, data from an experiment in which two types of information search tasks (simple and difficult), presented to both young and old participants was used. We found that in general difficult tasks demand significantly more time, significantly more clicks, significantly more reformulations and are answered significantly less accurately than simple tasks. Older persons inspect the search engine result pages significantly longer, produce significantly fewer reformulations with difficult tasks than younger persons, and are significantly more accurate than younger persons with simple tasks. We next used a cognitive model of web-navigation called CoLiDeS to predict which search engine result a user would choose to click. Old participants were found to click more often only on search engine results with high semantic similarity with the query. Search engine results generated by old participants were of higher semantic similarity value (computed w.r.t the query) than those generated by young participants only in the second cycle. Match between model-predicted clicks and actual user clicks was found to be significantly higher for difficult tasks compared to simple tasks. Potential improvements in enhancing the modeling and its applications are discussed.

Age-related Differences in the Content of Search Queries when Reformulating

This study investigated the change in the content of the queries when performing reformulations in relation to age and task difficulty. Results showed that both generalization and specialization strategies were applied significantly more often for difficult tasks compared to simple tasks. Young participants were found to use specialization strategy significantly more often than old participants. Generalization strategy was also used significantly more often by young participants, especially for difficult tasks. Young participants were found to reformulate much longer than old participants. The semantic relevance of queries with the target information was found to be significantly higher for difficult tasks compared to simple tasks. It showed a decreasing trend across reformulations for old participants and remained constant for young participants, indicating that as old participants reformulated, they produced queries that were further away from the target information. Implications of these findings for design of information search systems are discussed.

The role of content in addition to hyperlinks in user-clicking behavior

Motivation -- Cognitive models of web-navigation such as CoLiDeS, CoLiDeS+, SNIF-ACT compute the correct hyperlink by using information from the hyperlink text alone and ignore all other information on a web-page. This paper focuses on verifying the validity of this assumption by investigating the role played by the main content in addition to hyperlink text on the deciding the correct hyperlink. Research approach -- A mock-up website with two conditions: (i) with main content and hyperlinks and (ii) without main content but with hyperlinks was created. 18 students performed 8 information retrieval tasks on this mock-up website. Findings/Design -- The results showed that the user-click behaviour with or without main content remained largely the same. The same links were selected by users in both conditions. Also, the same amount of time was spent on the commonly selected links in both conditions. Research limitations/Implications -- We restrict ourselves to the role of main content in this experiment and did not study the impact of other factors like pictures. Originality/Value -- These results provide an empirical proof to the assumption CoLiDeS makes in its 3rd and 4th phases of focusing and selecting. Take away message -- Implication of the results is that one needs to study deeper the relevance/quality of wording used for hyperlinks in relation to the main content. We assume that if the wordings (of the links) are relevant or familiar to the user, the influence of main content would be negligible but if they are less relevant or unfamiliar, the content becomes more influential.

A STUDY ON THE ROLE OF NON-HYPERLINK TEXT ON WEB NAVIGATION

Cognitive models of web navigation have been used for evaluating websites and predicting user navigation behavior. Currently they predict the correct hyperlink by using information from the hyperlink text alone and ignore all other textual information on a webpage. The validity of this assumption is examined by investigating the role of non-hyperlink text on user navigation behavior. In the first experiment, we created two versions of a website by removing the non-hyperlink text from it. We found that there was no significant effect of non-hyperlink text on the user navigation behavior. Participants were equally accurate, selected the same set of pages to visit and spent the same amount of time on that common set with or without non-hyperlink text. This result validates the assumptions of those models of user-navigation behavior that consider information from the hyperlink text only. However, in a followup experiment, we included high-relevance and low-relevance pictures on the website, and repeated the experiment with and without non-hyperlink text. We found that participants were more accurate in the presence of non-hyperlink text than without it. This result suggests that the presence of pictures might prime the users to pay attention to non-hyperlink text, which increases the task accuracy.

Age-related effects of task difficulty on the semantic relevance of query reformulations

This study examined the semantics of query reformulations in relation to age and task difficulty. Task difficulty was manipulated using a metric called task preciseness defined as the semantic similarity of the task description with the content of the target page(s) containing the answer. A behavioral experiment was conducted in which 24 younger adults and 21 older adults solved six low precise and six high precise information search tasks. The behavioral outcomes were found to be in line with preceding work indicating that the metric was successful in differentiating different levels of task difficulty. Analysis of the semantic relevance of queries showed that for low precise tasks, the queries generated by younger adults had significantly higher mean semantic relevance than that of older adults whereas for high precise tasks, it was the other way round. When analyzed across reformulations, it was found that the mean semantic relevance of queries generated by older adults, decreased for both low and high precise tasks. For younger adults, it remained constant for high precise tasks and even increased for low precise tasks. Implications of these findings for the design of information search systems are discussed.