Han Weihong

Mining user interest in microblogs with a user-topic model

Microblogs have become an important platform for people to publish, transform information and acquire knowledge. This paper focuses on the problem of discovering user interest in microblogs. In this paper, we propose a topic mining model based on Latent Dirichlet Allocation (LDA) named user-topic model. For each user, the interests are divided into two parts by different ways to generate the microblogs: original interest and retweet interest. We represent a Gibbs sampling implementation for inference the parameters of our model, and discover not only users original interest, but also retweet interest. Then we combine original interest and retweet interest to compute interest words for users. Experiments on a dataset of Sina microblogs demonstrate that our model is able to discover user interest effectively and outperforms existing topic models in this task. And we find that original interest and retweet interest are similar and the topics of interest contain user labels. The interest words discovered by our model reflect user labels, but range is much broader.

Research and implement of real-time data loading system IMIL

With rapid development of the Internet and communication technology, massive data has been accumulated in many web-based applications such as deep web applications and web search engines. Increasing data volumes pose enormous challenges to data-loading techniques. This paper presents a data loading system in real time, the IMIL (Internet Monitoring Information Loader) that is used in RT-IMIS (Real-time Internet Monitoring Information System), which monitors real-time internet flux, manages network security, and collects a mass of Internet real-time information. IMIL consists of an extensible fault-tolerant hardware architecture, an efficient algorithm for bulk data loading using SQL*Loader and exchange partition mechanism, optimized parallelism, and guidelines for system tuning. Performance studies show the positive effects of these techniques with loading speed of every Cluster, increasing from 220 million records per day to 1.2 billion per day, and achieving the top loading speed of 6TB data when 10 Clusters are in parallel. This framework offers a promising approach for loading other large and complex databases.