Simpson Zhang

Distributed Online Learning in Social Recommender Systems

In this paper, we consider decentralized sequential decision making in distributed online recommender systems, where items are recommended to users based on their search query as well as their specific background including history of bought items, gender and age, all of which comprise the context information of the user. In contrast to centralized recommender systems, in which there is a single centralized seller who has access to the complete inventory of items as well as the complete record of sales and user information, in decentralized recommender systems each seller/learner only has access to the inventory of items and user information for its own products and not the products and user information of other sellers, but can get commission if it sells an item of another seller. Therefore, the sellers must distributedly find out for an incoming user which items to recommend (from the set of own items or items of another seller), in order to maximize the revenue from own sales and commissions. We formulate this problem as a cooperative contextual bandit problem, analytically bound the performance of the sellers compared to the best recommendation strategy given the complete realization of user arrivals and the inventory of items, as well as the context-dependent purchase probabilities of each item, and verify our results via numerical examples on a distributed data set adapted based on Amazon data. We evaluate the dependence of the performance of a seller on the inventory of items the seller has, the number of connections it has with the other sellers, and the commissions which the seller gets by selling items of other sellers to its users.

From Acquaintances to Friends: Homophily and Learning in Networks

This paper considers the evolution of a network in a discrete time, stochastic setting in which agents learn about each other through repeated interactions and maintain/break links on the basis of what they learn. Agents exhibit homophily, the preference to link with others who are similar to themselves, and they have a limited capacity for links. They thus maintain links with others learned to be similar to themselves and cut links to those learned to be dissimilar to themselves. We introduce a new equilibrium concept we term “matching pairwise stable equilibrium”, and we prove that such equilibrium is unique in our model. We show that higher levels of homophily decrease the (average) number of links that agents form. However, the effect of homophily is anomalous: mutually beneficial links may be dropped before learning is completed, thereby resulting in sparser networks and less clustering than under complete information. Homophily also exhibits an interesting interaction with the presence of incomplete information: initially, greater levels of homophily increase the difference between the complete and incomplete information networks, but sufficiently high levels of homophily eventually decrease the difference. Complete and incomplete information networks differ most when the degree of homophily is intermediate.

Multiagent Systems: Learning, Strategic Behavior, Cooperation, and Network Formation

Abstract Many applications ranging from crowdsourcing to recommender systems involve informationally decentralized agents repeatedly interacting with each other in order to reach their goals. These networked agents base their decisions on incomplete information, which they gather through interactions with their neighbors or through cooperation, which is often costly. This chapter presents a discussion on decentralized learning algorithms that enable the agents to achieve their goals through repeated interaction. First, we discuss cooperative online learning algorithms that help the agents to discover beneficial connections with each other and exploit these connections to maximize the reward. For this case, we explain the relation between the learning speed, network topology, and cooperation cost. Then, we focus on how informationally decentralized agents form cooperation networks through learning. We explain how learning features prominently in many real-world interactions, and greatly affects the evolution of social networks. Links that otherwise would not have formed may now appear, and a much greater variety of network configurations can be reached. We show that the impact of learning on efficiency and social welfare could be both positive or negative. We also demonstrate the use of the aforementioned methods in popularity prediction, recommender systems, expert selection, and multimedia content aggregation.

The Population Dynamics of Websites: [Extended Abstract]

Websites derive revenue by advertising or charging fees for services and so their profit depends on their user base -- the number of users visiting the website. But how should websites control their user base? This paper is the first to address and answer this question. It builds a model in which, starting from an initial user base, the website controls the growth of the population by choosing the intensity of referrals and targeted ads to potential users. A larger population provides more profit to the website, but building a larger population through referrals and targeted ads is costly; the optimal policy must therefore balance the marginal benefit of adding users against the marginal cost of referrals and targeted ads. The nature of the optimal policy depends on a number of factors. Most obvious is the initial user base; websites starting with a small initial population should offer many referrals and targeted ads at the beginning, but then decrease referrals and targeted ads over time. Less obvious factors are the type of website and the typical length of time users remain on the site: the optimal policy for a website that generates most of its revenue from a core group of users who remain on the site for a long time -- e.g., mobile and online gaming sites -- should be more aggressive and protective of its user base than that of a website whose revenue is more uniformly distributed across users who remain on the site only briefly. When arrivals and exits are stochastic, the optimal policy is more aggressive -- offering more referrals and targeted ads.

Towards a theory of societal co-evolution: Individualism versus collectivism

Substantial empirical research has shown that the level of individualism vs. collectivism is one of the most critical and important determinants of societal traits, such as economic growth, economic institutions and health conditions. But the exact nature of this impact has thus far not been well understood in an analytical setting. In this work, we develop one of the first theoretical models that analytically studies the impact of individualism-collectivism on the society. We model the growth of an individuals welfare (wealth, resources and health) as depending not only on himself, but also on the level of collectivism, i.e. the level of dependence on the rest of the individuals in the society, which leads to a co-evolutionary setting. Based on our model, we are able to predict the impact of individualism-collectivism on various societal metrics, such as average welfare, average lifetime, total population, cumulative welfare and average inequality. We analytically show that individualism has a positive impact on average welfare and cumulative welfare, but comes with the drawbacks of lower average life-time, lower total population and higher average inequality.

Network evolution with incomplete information and learning

We analyze networks that feature reputational learning: how links are initially formed by agents under incomplete information, how agents learn about their neighbors through these links, and how links may ultimately become broken. We show that the type of information agents have access to, and the speed at which agents learn about each other, can have tremendous repercussions for the network evolution and the overall network social welfare. Specifically, faster learning can often be harmful for networks as a whole if agents are myopic, because agents fail to fully internalize the benefits of experimentation and break off links too quickly. As a result, preventing two agents from linking with each other can be socially beneficial, even if the two agents are initially believed to be of high quality. This is due to the fact that having fewer connections slows the rate of learning about these agents, which can be socially beneficial. Another method of solving the informational problem is to impose costs for breaking links, in order to incentivize agents to experiment more carefully.