Qi Gu

Automatic content based image retrieval using semantic analysis

We present a new text-to-image re-ranking approach for improving the relevancy rate in searches. In particular, we focus on the fundamental semantic gap that exists between the low-level visual features of the image and high-level textual queries by dynamically maintaining a connected hierarchy in the form of a concept database. For each textual query, we take the results from popular search engines as an initial retrieval, followed by a semantic analysis to map the textual query to higher level concepts. In order to do this, we design a two-layer scoring system which can identify the relationship between the query and the concepts automatically. We then calculate the image feature vectors and compare them with the classifier for each related concept. An image is relevant only when it is related to the query both semantically and content-wise. The second feature of this work is that we loosen the requirement for query accuracy from the user, which makes it possible to perform well on users’ queries containing less relevant information. Thirdly, the concept database can be dynamically maintained to satisfy the variations in user queries, which eliminates the need for human labor in building a sophisticated initial concept database. We designed our experiment using complex queries (based on five scenarios) to demonstrate how our retrieval results are a significant improvement over those obtained from current state-of-the-art image search engines.

Incomplete information and Bayesian Knowledge-Bases

Knowledge acquisition is an essential process in improving the problem-solving capabilities of existing knowledge-based systems through the absorption of new information and facilitating change in current knowledge. However, without a verification mechanism, these changes could result in violations of semantic soundness of the knowledge causing inconsistencies and ultimately, contradictions. Therefore, maintaining semantic consistency is of primary concern, especially when dealing with incompleteness and uncertainty. In this paper, we consider the semantic completability of a knowledge system as a means of ensuring long-term semantic soundness. In particular, we focus on how to preserve semantic completability as the knowledge evolves over time. Among numerous methods of knowledge representation under uncertainty, we examine Bayesian Knowledge-Bases, which are a rule-based probabilistic model that allows for incompleteness and cycles between variables. A formal definition of full/partial completability of BKB is first introduced. A principle to check the overall completability of a BKB is then formulated with a formal proof of correctness. Furthermore, we show how to use this principle as a guide for maintaining semantic soundness and completability during incremental knowledge acquisition. In particular, we consider two primary modifications to the knowledge base: 1) adding/fusing knowledge, and 2) changing/tuning conditional probabilities.

Incorporating Social Theories in Computational Behavioral Models

Computational social science methodologies are increasingly being viewed as critical for modeling complex individual and organizational behaviors in dynamic, real world scenarios. However, many challenges for identifying, representing and incorporating appropriate socio-cultural behaviors remain. Social theories provide rules, which have strong theoretic underpinnings and have been empirically validated, for representing and analyzing individual and group interactions. The key insight in this paper is that social theories can be embedded into computational models as functional mappings based on underlying factors, structures and interactions in social systems. We describe a generic framework, called a Culturally Infused Social Network (CISN), which makes such mappings realizable with its abilities to incorporate multi-domain socio-cultural factors, model at multiple scales, and represent dynamic information. We explore the incorporation of different social theories for added rigor to modeling and analysis by analyzing the fall of the Islamic Courts Union (ICU) regime in Somalia during the latter half of 2006. Specifically, we incorporate the concepts of homophily and frustration to examine the strength of the ICU’s alliances during its rise and fall. Additionally, we employ Affect Control Theory (ACT) to improve the resolution and detail of the model, and thus enhance the explanatory power of the CISN framework.

Modeling emergent border-crossing behaviors during pandemics

Modeling real-world scenarios is a challenge for traditional social science researchers, as it is often hard to capture the intricacies and dynamisms of real-world situations without making simplistic assumptions. This imposes severe limitations on the capabilities of such models and frameworks. Complex population dynamics during natural disasters such as pandemics is an area where computational social science can provide useful insights and explanations. In this paper, we employ a novel intent-driven modeling paradigm for such real-world scenarios by causally mapping beliefs, goals, and actions of individuals and groups to overall behavior using a probabilistic representation called Bayesian Knowledge Bases (BKBs). To validate our framework we examine emergent behavior occurring near a national border during pandemics, specifically the 2009 H1N1 pandemic in Mexico. The novelty of the work in this paper lies in representing the dynamism at multiple scales by including both coarse-grained (events at the national level) and finegrained (events at two separate border locations) information. This is especially useful for analysts in disaster management and first responder organizations who need to be able to understand both macro-level behavior and changes in the immediate vicinity, to help with planning, prevention, and mitigation. We demonstrate the capabilities of our framework in uncovering previously hidden connections and explanations by comparing independent models of the border locations with their fused model to identify emergent behaviors not found in either independent location models nor in a simple linear combination of those models.

Bayesian knowledge base tuning

Abstract For a knowledge-based system that fails to provide the correct answer, it is important to be able to tune the system while minimizing overall change in the knowledge-base. There are a variety of reasons why the answer is incorrect ranging from incorrect knowledge to information vagueness to incompleteness. Still, in all these situations, it is typically the case that most of the knowledge in the system is likely to be correct as specified by the expert (s) and/or knowledge engineer (s). In this paper, we propose a method to identify the possible changes by understanding the contribution of parameters on the outputs of concern. Our approach is based on Bayesian Knowledge Bases for modeling uncertainties. We start with single parameter changes and then extend to multiple parameters. In order to identify the optimal solution that can minimize the change to the model as specified by the domain experts, we define and evaluate the sensitivity values of the results with respect to the parameters. We discuss the computational complexities of determining the solution and show that the problem of multiple parameters changes can be transformed into Linear Programming problems, and thus, efficiently solvable. Our work can also be applied towards validating the knowledge base such that the updated model can satisfy all test-cases collected from the domain experts.

Hidden Source Behavior Change Tracking and Detection

An important task of modeling complex social behaviors is to observe and understand individual/group beliefs and attitudes. These beliefs, however, are not stable and may change multiple times as people gain additional information/perceptions from various external sources, which in turn, may affect their subsequent behavior. To detect and track such influential sources is challenging, as they are often invisible to the public due to a variety of reasons -- private communications, what one randomly reads or hears, and implicit social hierarchies, to name a few. Existing approaches usually focus on detecting distribution variations in behavioral data, but overlook the underlying reason for the variation. In this paper, we present a novel approach that models the belief change over time caused by hidden sources, taking into consideration the evolution of their impact patterns. Specifically, a finite fusion model is defined to encode the latent parameters that characterize the distribution of the hidden sources and their impact weights. We compare our work with two general mixture models, namely Gaussian Mixture Model and Mixture Bayesian Network. Experiments on both synthetic data and a real-world scenario show that our approach is effective on detecting and tracking hidden sources and outperforms existing methods.

Modeling Opinion Dynamics in a Social Network

Opinion dynamics is a complex procedure that entails a cognitive process when dealing with how a person integrates influential opinions to form a revised opinion. In this work, we present a new approach to model opinion dynamics by treating the opinion on an issue as a product inferred from ones knowledge bases, where the knowledge bases keep growing and updating through social interaction. A general impact metric is proposed to evaluate the likelihood of a person adopting the opinions from others. Specifically, a set of domain-independent influential factors is selected based on social and communication theories, but the weights of these factors are missing. Though the opinions from different actors are not integrated linearly like traditional methods, we show that the factor weights can be efficiently learned via regression. We validated the effectiveness of our model by comparing against a baseline model on both synthetic and real datasets. The contribution of this paper lies with 1) a novel opinion dynamics model that emphasize the dependencies between knowledge pieces, 2) proof that the classical DeGroot model is a special case of our model under certain conditions, and, 3) to the best of our knowledge, this is the first work to try and uncover the mechanism that guides the selection of opinions in the real world by modeling opinion change.

Modeling Socio-Cultural Processes in Network Centric Environments

The major focus in the field of modeling & simulation for network centric environments has been on the physical layer while making simplifications for the human-in-the-loop. However, the human element has a big impact on the capabilities of network centric systems. Taking into account the socio-behavioral aspects of processes such as team building, group decision-making, etc. are critical to realistically modeling and analyzing system performance. Modeling socio-cultural processes is a challenge because of the complexity of the networks, dynamism in the physical and social layers, feedback loops and uncertainty in the modeling data. We propose an overarching framework to represent, model and analyze various socio-cultural processes within network centric environments. The key innovation in our methodology is to simultaneously model the dynamism in both the physical and social layers while providing functional mappings between them. We represent socio-cultural information such as friendships, professional relationships and temperament by leveraging the Culturally Infused Social Network (CISN) framework. The notion of intent is used to relate the underlying socio-cultural factors to observed behavior. We will model intent using Bayesian Knowledge Bases (BKBs), a probabilistic reasoning network, which can represent incomplete and uncertain socio-cultural information. We will leverage previous work on a network performance modeling framework called Network-Centric Operations Performance and Prediction (N-COPP) to incorporate dynamism in various aspects of the physical layer such as node mobility, transmission parameters, etc. We validate our framework by simulating a suitable scenario, incorporating relevant factors and providing analyses of the results.