Thomas George Kannampallil

Evaluating tagging behavior in social bookmarking systems: metrics and design heuristics

To improve existing social bookmarking systems and to design new ones, researchers and practitioners need to understand how to evaluate tagging behavior. In this paper, we analyze over two years of data from CiteULike, a social bookmarking system for tagging academic papers. We propose six tag metrics-tag growth, tag reuse, tag non-obviousness, tag discrimination, tag frequency, and tag patterns-to understand the characteristics of a social bookmarking system. Using these metrics, we suggest possible design heuristics to implement a social bookmarking system for CiteSeer, a popular online scholarly digital library for computer science. We believe that these metrics and design heuristics can be applied to social bookmarking systems in other domains.

A systematic review of the literature on the evaluation of handoff tools: implications for research and practice

OBJECTIVE Given the complexities of the healthcare environment, efforts to develop standardized handoff practices have led to widely varying manifestations of handoff tools. A systematic review of the literature on handoff evaluation studies was performed to investigate the nature, methodological, and theoretical foundations underlying the evaluation of handoff tools and their adequacy and appropriateness in achieving standardization goals. METHOD We searched multiple databases for articles evaluating handoff tools published between 1 February 1983 and 15 June 2012. The selected articles were categorized along the following dimensions: handoff tool characteristics, standardization initiatives, methodological framework, and theoretical perspectives underlying the evaluation. RESULTS Thirty-six articles met our inclusion criteria. Handoff evaluations were conducted primarily on electronic tools (64%), with a more recent focus on electronic medical record-integrated tools (36% since 2008). Most evaluations centered on intra-departmental tools (95%). Evaluation studies were quasi-experimental (42%) or observational (50%), with a major focus on handoff-related outcome measures (94%) using predominantly survey-based tools (70%) with user satisfaction metrics (53%). Most of the studies (81%) based their evaluation on aspects of standardization that included continuity of care and patient safety. CONCLUSIONS The nature, methodological, and theoretical foundations of handoff tool evaluations varied significantly in terms of their quality and rigor, thereby limiting their ability to inform strategic standardization initiatives. Future research should utilize rigorous, multi-method qualitative and quantitative approaches that capture the contextual nuances of handoffs, and evaluate their effect on patient-related outcomes.

Semantic imitation in social tagging

We present a semantic imitation model of social tagging and exploratory search based on theories of cognitive science. The model assumes that social tags evoke a spontaneous tag-based topic inference process that primes the semantic interpretation of resource contents during exploratory search, and the semantic priming of existing tags in turn influences future tag choices. The model predicts that (1) users who can see tags created by others tend to create tags that are semantically similar to these existing tags, demonstrating the social influence of tag choices; and (2) users who have similar information goals tend to create tags that are semantically similar, but this effect is mediated by the semantic representation and interpretation of social tags. Results from the experiment comparing tagging behavior between a social group (where participants can see tags created by others) and a nominal group (where participants cannot see tags created by others) confirmed these predictions. The current results highlight the critical role of human semantic representations and interpretation processes in the analysis of large-scale social information systems. The model implies that analysis at both the individual and social levels are important for understanding the active, dynamic processes between human knowledge structures and external folksonomies. Implications on how social tagging systems can facilitate exploratory search, interactive information retrievals, knowledge exchange, and other higher-level cognitive and learning activities are discussed.

Adaptive information search: age-dependent interactions between cognitive profiles and strategies

Previous research has shown that older adults performed worse in web search tasks, and attributed poorer performance to a decline in their cognitive abilities. We conducted a study involving younger and older adults to compare their web search behavior and performance in ill-defined and well-defined information tasks using a health information website. In ill-defined tasks, only a general description about information needs was given, while in well-defined tasks, information needs as well as the specific target information were given. We found that older adults performed worse than younger adults in well-defined tasks, but the reverse was true in ill-defined tasks. Older adults compensated for their lower cognitive abilities by adopting a top-down knowledge-driven strategy to achieve the same level of performance in the ill-defined tasks. Indeed, path models showed that cognitive abilities, health literacy, and knowledge influenced search strategies adopted by older and younger adults. Design implications are also discussed.

A Semantic Imitation Model of Social Tag Choices

We describe a semantic imitation model of social tagging that integrates formal representations of semantics and a stochastic tag choice process to explain and predict emergent behavioral patterns. The model adopts a probabilistic topic model to separately represent external word-topic and internal word-concept relations. These representations are coupled with a tag-based topic inference process that predicts how existing tags may influence the semantic interpretation of a document. The inferred topics influence the choice of tags assigned to a document through a random utility model of tag choices. We show that the model is successful in explaining the stability in tag proportions across time and power-law frequency-rank distributions of tag co-occurrences for semantically general and narrow tags. The model also generates novel predictions on how emergent behavioral patterns may change when users with different domain expertise interact with a social tagging system. The model demonstrates the weaknesses of single-level analyses and highlights the importance of adopting a multi-level modeling approach to explain online social behavior.

Comparative evaluation of the content and structure of communication using two handoff tools: Implications for patient safety

PURPOSE Handoffs vary in their structure and content, raising concerns regarding standardization. We conducted a comparative evaluation of the nature and patterns of communication on 2 functionally similar but conceptually different handoff tools: Subjective, Objective, Assessment and Plan, based on a patient problem-based format, and Handoff Intervention Tool (HAND-IT), based on a body system-based format. METHOD A nonrandomized pre-post prospective intervention study supported by audio recordings and observations of 82 resident handoffs was conducted in a medical intensive care unit. Qualitative analysis was complemented with exploratory sequential pattern analysis techniques to capture the characteristics and types of communication events (CEs) and breakdowns. RESULTS Use of HAND-IT led to fewer communication breakdowns (F1,80 = 45.66: P < .0001), greater number of CEs (t40 = 4.56; P < .001), with more ideal CEs than Subjective, Objective, Assessment and Plan (t40 = 9.27; P < .001). In addition, the use of HAND-IT was characterized by more request-response CE transitions. CONCLUSION The HAND-ITs body system-based structure afforded physicians the ability to better organize and comprehend patient information and led to an interactive and streamlined communication, with limited external input. Our results also emphasize the importance of information organization using a medical knowledge hierarchical format for fostering effective communication.

Making sense: Sensor-based investigation of clinician activities in complex critical care environments

In many respects, the critical care workplace resembles a paradigmatic complex system: on account of the dynamic and interactive nature of collaborative clinical work, these settings are characterized by non-linear, inter-dependent and emergent activities. Developing a comprehensive understanding of the work activities in critical care settings enables the development of streamlined work practices, better clinician workflow and most importantly, helps in the avoidance of and recovery from potential errors. Sensor-based technology provides a flexible and viable way to complement human observations by providing a mechanism to capture the nuances of certain activities with greater precision and timing. In this paper, we use sensor-based technology to capture the movement and interactions of clinicians in the Trauma Center of an Emergency Department (ED). Remarkable consistency was found between sensor data and human observations in terms of clinician locations and interactions. With this validation and greater precision with sensors, ED environment was characterized in terms of (a) the degree of randomness or entropy in the environment, (b) the movement patterns of clinicians, (c) interactions with other clinicians and finally, (d) patterns of collaborative organization with team aggregation and dispersion. Based on our results, we propose three opportunities for the use of sensor technologies in critical care settings: as a mechanism for real-time monitoring and analysis for ED activities, education and training of clinicians, and perhaps most importantly, investigating the root-causes, origins and progression of errors in the ED. Lessons learned and the challenges encountered in designing and implementing the sensor technology sensor data are discussed.

Cognitive informatics in biomedicine and healthcare

Cognitive Informatics (CI) is a burgeoning interdisciplinary domain comprising of the cognitive and information sciences that focuses on human information processing, mechanisms and processes within the context of computing and computer applications. Based on a review of articles published in the Journal of Biomedical Informatics (JBI) between January 2001 and March 2014, we identified 57 articles that focused on topics related to cognitive informatics. We found that while the acceptance of CI into the mainstream informatics research literature is relatively recent, its impact has been significant - from characterizing the limits of clinician problem-solving and reasoning behavior, to describing coordination and communication patterns of distributed clinical teams, to developing sustainable and cognitively-plausible interventions for supporting clinician activities. Additionally, we found that most research contributions fell under the topics of decision-making, usability and distributed team activities with a focus on studying behavioral and cognitive aspects of clinical personnel, as they performed their activities or interacted with health information systems. We summarize our findings within the context of the current areas of CI research, future research directions and current and future challenges for CI researchers.

Diagnostic Reasoning and Decision Making in the Context of Health Information Technology

Diagnostic reasoning and medical decision making have been focal areas of research in the fields of medical education, cognition, and artificial intelligence in medicine. Drawing on several decades worth of research, we propose an integrated summary of prior research on diagnostic reasoning and decision making—in terms of both historical development and theoretical shifts. We also characterize the changes in research and theory resulting from the incorporation and adoption of health information technology in the clinical work place. In this paper, we differentiate between the various forms of diagnostic reasoning and trace the evolution of the various models of reasoning, including knowledge-based, exemplar-based, and visual strategies. We also discuss the effect of clinical expertise on reasoning processes. Within the medical decision-making research, we delineate the various approaches highlighting decision-making errors that arise due to the nature of heuristics and biases and other factors. Although t...

Role of cognition in generating and mitigating clinical errors.

Given the complexities of current clinical practice environments, strategies to reduce clinical error must appreciate that error detection and recovery are integral to the function of complex cognitive systems. In this review, while acknowledging that error elimination is an attractive notion, we use evidence to show that enhancing error detection and improving error recovery are also important goals. We further show how departures from clinical protocols or guidelines can yield innovative and appropriate solutions to unusual problems. This review addresses cognitive approaches to the study of human error and its recovery process, highlighting their implications in promoting patient safety and quality. In addition, we discuss methods for enhancing error recognition, and promoting suitable responses, through external cognitive support and virtual reality simulations for the training of clinicians.