Xiaozhong Liu

The distribution of references across texts: Some implications for citation analysis

In citation network analysis, complex behavior is reduced to a simple edge, namely, node A cites node B. The implicit assumption is that A is giving credit to, or acknowledging, B. It is also the case that the contributions of all citations are treated equally, even though some citations appear multiply in a text and others appear only once. In this study, we apply text-mining algorithms to a relatively large dataset (866 information science articles containing 32,496 bibliographic references) to demonstrate the differential contributions made by references. We (1) look at the placement of citations across the different sections of a journal article, and (2) identify highly cited works using two different counting methods (CountOne and CountX). We find that (1) the most highly cited works appear in the Introduction and Literature Review sections of citing papers, and (2) the citation rankings produced by CountOne and CountX differ. That is to say, counting the number of times a bibliographic reference is cited in a paper rather than treating all references the same no matter how many times they are invoked in the citing article reveals the differential contributions made by the cited works to the citing paper.

Full-text citation analysis: A new method to enhance scholarly networks

In this article, we use innovative full‐text citation analysis along with supervised topic modeling and network‐analysis algorithms to enhance classical bibliometric analysis and publication/author/venue ranking. By utilizing citation contexts extracted from a large number of full‐text publications, each citation or publication is represented by a probability distribution over a set of predefined topics, where each topic is labeled by an author‐contributed keyword. We then used publication/citation topic distribution to generate a citation graph with vertex prior and edge transitioning probability distributions. The publication importance score for each given topic is calculated by PageRank with edge and vertex prior distributions. To evaluate this work, we sampled 104 topics (labeled with keywords) in review papers. The cited publications of each review paper are assumed to be “important publications” for the target topic (keyword), and we use these cited publications to validate our topic‐ranking result and to compare different publication‐ranking lists. Evaluation results show that full‐text citation and publication content prior topic distribution, along with the classical PageRank algorithm can significantly enhance bibliometric analysis and scientific publication ranking performance, comparing with term frequency–inverted document frequency (tf–idf), language model, BM25, PageRank, and PageRank + language model (p

Full-text based context-rich heterogeneous network mining approach for citation recommendation

Citation relationship between scientific publications has been successfully used for scholarly bibliometrics, information retrieval and data mining tasks, and citation-based recommendation algorithms are well documented. While previous studies investigated citation relations from various viewpoints, most of them share the same assumption that, if paper1 cites paper2 (or author1 cites author2), they are connected, regardless of citation importance, sentiment, reason, topic, or motivation. However, this assumption is oversimplified. In this study, we employ an innovative “context-rich heterogeneous network” approach, which paves a new way for citation recommendation task. In the network, we characterize (1) the importance of citation relationships between citing and cited papers, and (2) the topical citation motivation. Unlike earlier studies, the citation information, in this paper, is characterized by citation textual contexts extracted from the full-text citing paper. We also propose algorithm to cope with the situation when large portion of full-text missing information exists in the bibliographic repository. Evaluation results show that, context-rich heterogeneous network can significantly enhance the citation recommendation performance.

Generating metadata for cyberlearning resources through information retrieval and meta-search

The goal of this study was to propose novel cyberlearning resource‐based scientific referential metadata for an assortment of publications and scientific topics, in order to enhance the learning experiences of students and scholars in a cyberinfrastructure‐enabled learning environment. By using information retrieval and meta‐search approaches, different types of referential metadata, such as related Wikipedia pages, data sets, source code, video lectures, presentation slides, and (online) tutorials for scientific publications and scientific topics will be automatically retrieved, associated, and ranked. In order to test our method of automatic cyberlearning referential metadata generation, we designed a user experiment to validate the quality of the metadata for each scientific keyword and publication and resource‐ranking algorithm. Evaluation results show that the cyberlearning referential metadata retrieved via meta‐search and statistical relevance ranking can help students better understand the essence of scientific keywords and publications.

Predicting popularity of online distributed applications: iTunes app store case analysis

Online distributed applications are becoming more and more important for users nowadays. There are an increasing number of individuals and companies developing applications and selling them online. In the past couple of years, Apple Inc. has successfully built an online application distribution platform -- iTunes App Store, which is facilitated by their fashionable hardware such like iPad or iPhone. Unlike other traditional selling networks, iTunes has some unique features to advertise their application, for example, daily application ranking, application recommendation, free trial application usage, application update, and user comments. All of these make us wonder what makes an application popular in the iTunes store and why users are interested in some specific type of applications. We plan to answer these questions by using machine learning techniques.

Real‐time user interest modeling for real‐time ranking

User interest as a very dynamic information need is often ignored in most existing information retrieval systems. In this research, we present the results of experiments designed to evaluate the performance of a real‐time interest model (RIM) that attempts to identify the dynamic and changing query level interests regarding social media outputs. Unlike most existing ranking methods, our ranking approach targets calculation of the probability that user interest in the content of the document is subject to very dynamic user interest change. We describe 2 formulations of the model (real‐time interest vector space and real‐time interest language model) stemming from classical relevance ranking methods and develop a novel methodology for evaluating the performance of RIM using Amazon Mechanical Turk to collect (interest‐based) relevance judgments on a daily basis. Our results show that the model usually, although not always, performs better than baseline results obtained from commercial web search engines. We identify factors that affect RIM performance and outline plans for future research.

Automatic Feature Generation on Heterogeneous Graph for Music Recommendation

Online music streaming services (MSS) experienced exponential growth over the past decade. The giant MSS providers not only built massive music collection with metadata, they also accumulated large amount of heterogeneous data generated from users, e.g. listening history, comment, bookmark, and user generated playlist. While various kinds of user data can potentially be used to enhance the music recommendation performance, most existing studies only focused on audio content features and collaborative filtering approaches based on simple user listening history or music rating. In this paper, we propose a novel approach to solve the music recommendation problem by means of heterogeneous graph mining. Meta-path based features are automatically generated from a content-rich heterogeneous graph schema with 6 types of nodes and 16 types of relations. Meanwhile, we use learning-to-rank approach to integrate different features for music recommendation. Experiment results show that the automatically generated graphical features significantly (p<0.0001) enhance state-of-the-art collaborative filtering algorithm.

Scientific Metadata Quality Enhancement for Scholarly Publications

Keyword metadata is very important to the access, retrieval, and management of scientific publications. However, author-assigned keywords are not always readily available in digital repositories. In this study, in order to enhance metadata quality, we explore different automatic methods to infer keywords from scholarly articles, including supervised topic modeling, language model, and mutual information. Evaluation results showed that the linear combination of mutual information and topic modeling with full text outperform other methods on MAP, while language model with abstract performed better than other methods on the measure of precision@10.

Finding topic-level experts in scholarly networks

Expert finding is of vital importance for exploring scientific collaborations to increase productivity by sharing and transferring knowledge within and across different research areas. Expert finding methods, including content-based methods, link structure-based methods, and a combination of content-based and link structure-based methods, have been studied in recent years. However, most state-of-the-art expert finding approaches have usually studied candidates’ personal information (e.g. topic relevance and citation counts) and network information (e.g. citation relationship) separately, causing some potential experts to be ignored. In this paper, we propose a topical and weighted factor graph model that simultaneously combines all the possible information in a unified way. In addition, we also design the Loopy Max-Product algorithm and related message-passing schedules to perform approximate inference on our cycle-containing factor graph model. Information Retrieval is chosen as the test field to identify representative authors for different topics within this area. Finally, we compare our approach with three baseline methods in terms of topic sensitivity, coverage rate of SIGIR PC (e.g. Program Committees or Program Chairs) members, and Normalized Discounted Cumulated Gain scores for different rankings on each topic. The experimental results demonstrate that our factor graph-based model can definitely enhance the expert-finding performance.

Scientific Information Understanding via Open Educational Resources (OER)

Scientific publication retrieval/recommendation has been investigated in the past decade. However, to the best of our knowledge, few efforts have been made to help junior scholars and graduate students to understand and consume the essence of those scientific readings. This paper proposes a novel learning/reading environment, OER-based Collaborative PDF Reader (OCPR), that incorporates innovative scaffolding methods that can: 1. auto-characterize student emerging information need while reading a paper; and 2. enable students to readily access open educational resources (OER) based on their information need. By using metasearch methods, we pre-indexed 1,112,718 OERs, including presentation videos, slides, algorithm source code, or Wikipedia pages, for 41,378 STEM publications. Based on the computational information need, we use text mining and heterogeneous graph mining algorithms to recommend high quality OERs to help students better understand the scientific content in the paper. Evaluation results and exit surveys for an information retrieval course show that the OCPR system alone with the recommended OERs can effectively assist graduate students better understand the complex STEM publications. For instance, 78.42% of participants believe the OCPR system and recommended OERs can provide precise and useful information they need, while 78.43% of them believe the recommended OERs are close to exactly what they need when reading the paper. From OER ranking viewpoint, MRR, MAP and NDCG results prove that learning to rank and cold start solutions can efficiently integrate different text and graph ranking features.