Vijay Kamble

Evolutionary forwarding games in delay tolerant networks: Equilibria, mechanism design and stochastic approximation

In this paper, we apply evolutionary games to non-cooperative forwarding control in Delay Tolerant Networks (DTNs). The main focus is on mechanisms to rule the participation of the relays to the delivery of messages in DTNs. Thus, we express the success probability as a function of the competition that takes place within a large population of mobiles, and we characterize the effect of reward-based mechanisms on the performance of such systems. Devices acting as active relays, in fact, sacrifice part of their batteries in order to support message replication and thus increase the probability to reach the destination. In our scheme, a relay can choose the strategy by which they participate to the message relaying. A mobile that participates receives a unit of reward based on the reward mechanism selected by the network. A utility function is introduced as the difference between the expected reward and the energy cost, i.e., the cost spent by the relay to sustain forwarding operations. We show how the evolution dynamics and the equilibrium behavior (called Evolutionary Stable Strategy - ESS) are influenced by the characteristics of inter contact time, energy expenditure and pricing characteristics. We extend our analysis to mechanisms that the system can introduce in order to have the message delivered to the destination with high probability within a given deadline and under energy constraints which bound the number of released copies per message. Finally, we apply our findings in order to devise decentralized forwarding algorithms that are rooted in the theory of stochastic approximations. Thus, we demonstrate that the ESS can be attained without complete knowledge of the system state and letting the source monitor number of released copies per message only. We provide extensive numerical results to validate the proposed scheme.

Risk sensitive optimal control framework applied to delay tolerant networks

Epidemics dynamics can describe the dissemination of information in delay tolerant networks, in peer to peer networks and in content delivery networks. The control of such dynamics has thus gained a central role in all of these areas. However, a major difficulty in this context is that the objective functions to be optimized are often not additive in time but are rather multiplicative. The classical objective function in DTNs, i.e., the successful delivery probability of a message within a given deadline, falls precisely in this category, because it takes often the form of the expectation of the exponent of some integral cost. So far, models involving such costs have been solved by interchanging the order of expectation and the exponential function. While reducing the problem to a standard optimal control problem, this interchange is only tight in the mean field limit obtained as the population tends to infinity. In this paper we identify a general framework from optimal control in finance, known as risk sensitive control, which let us handle the original (multiplicative) cost and obtain solutions to several novel control problems in DTNs. In particular, we can derive the structure of state-dependent controls that optimize transmission power at the source node. Further, we can account for the propagation loss factor of the wireless medium while obtaining these controls, and, finally, we address power control at the destination node, resulting in a novel threshold optimal activation policy. Combined optimal power control at source and destination nodes is also obtained.

Matching while Learning

We consider the problem faced by a service platform that needs to match supply with demand but also to learn attributes of new arrivals in order to match them better in the future. We introduce a benchmark model with heterogeneous workers and jobs that arrive over time. Job types are known to the platform, but worker types are unknown and must be learned by observing match outcomes. Workers depart after performing a certain number of jobs. The payoff from a match depends on the pair of types and the goal is to maximize the steady-state rate of accumulation of payoff. Our main contribution is a complete characterization of the structure of the optimal policy in the limit that each worker performs many jobs. The platform faces a trade-off for each worker between myopically maximizing payoffs (exploitation) and learning the type of the worker (exploration). This creates a multitude of multi-armed bandit problems, one for each worker, coupled together by the constraint on the availability of jobs of different types (capacity constraints). We find that the platform should estimate a shadow price for each job type, and use the payoffs adjusted by these prices, first, to determine its learning goals and then, for each worker, (i) to balance learning with payoffs during the exploration phase, and (ii) to myopically match after it has achieved its learning goals during the exploitation phase.

Strategyproof Mechanisms for Purchasing a Shared Resource

Consider a situation where a group of buyers would like to jointly purchase a particular resource with the intention of sharing it. For example, suppose two individuals who are sharing a living space would like to purchase an object, say an air conditioner or a television, which is available in the market for a certain price. How do they agree upon a division of this price amongst themselves when their utilties for using that object are private? Any scheme that recommends some notion of fair division of this price has to rely on the ability of the scheme to elicit the true utilities of the individuals, which is difficult since each individual wants to minimize his share of the payment. More generally, the resource in question may be congestible, and the utilities may depend on the proportion in which it is shared between the two users. In that case they not only have to decide how to divide the price of the resource but also how it will be shared, and the two decisions would naturally have to go hand in hand. Moreover, it may not be a simple question of paying a given price, but the resource itself may be offered in an auction, in which case the two buyers need to decide how they will jointly bid in the auction, along with the terms of sharing the resource and the division of payment in the event that they win. Embracing the classical perspective of mechanism design, we can transfer the onus of coming up with a solution to this problem from the buyers to the seller herself. This leads us to consider the converse problem from the perspective of the seller in the market. She intends to sell a resource and several competing groups of buyers are interested in purchasing that resource for their respective groups. Her problem is thus to design a mechanism to allocate the resource to one of the competing groups, along with a proposed division of the resource within the group. The key aspect of this design problem is the kind of incentive properties such a mechanism needs to satisfy. The buyers in a single group are expected to collude in their utility reports and hence such a mechanism needs to be robust to any collusive behavior within a particular group, but perhaps not necessarily across groups. A practical example, which is our primary motivation for studying this kind of a market, is the market for radio spectrum. Recently there has been a debate concerning the merits and demerits of allocating newly opened blocks of spectrum for free unlicensed use (like WiFi) as opposed to selling them for exclusive licensed use (e.g., to cellular ser-

A Theoretical Framework for Hierarchical Routing Games

Most theoretical research on routing games in telecommunication networks has so far dealt with reciprocal congestion effects between routed entities. Yet in networks that support differentiation between flows, the congestion experienced by a packet depends on its priority level. Another differentiation is made by compressing the packets in the low priority flow while leaving the high priority flow intact. In this paper we study such kind of routing scenarios for the case of non-atomic users and we establish conditions for the existence and uniqueness of equilibrium.

Network Formation of Coalition Loyalty Programs

Loyalty programs are widely used in consumer retail, primarily serving as a mechanism for customer acquisition and retention [14, 15]. The most common forms of loyalty programs are frequency reward programs, such as airline frequent flyer programs, wherein customers earn certain number of points from every purchase. These points can be subsequently redeemed for rewards. The value of these rewards typically increases faster with the number of points – this non-linearity in rewards is what incentivizes members of a program to become more loyal to the merchant. Over time, many stand-alone loyalty programs have agglomerated into larger coalition programs which allow customers to earn and redeem points across merchant partners in the coalition at specified exchange rates. The observed coalition networks are surprisingly complex (cf. Fig. 1), encompassing both pairwise partnerships as well as more centralized coalition loyalty programs such as Star Alliance and OneWorld (international airline alliances), Nectar (U.K.), Air Miles (Canada), Payback (Germany), Fly Buys (Australia), etc. Moreover, there is great variety in the exchange rates observed between different programs. Though coalition loyalty programs are very popular and well studied in the literature [4, 3, 12], there is little formal understanding of the structure and strategic formation of such networks, specifically the exchange rates between different programs. Our work aims to address this gap. Modeling Coalition Loyalty Programs: A network of loyalty programs can be viewed as a weighted directed graph, with nodes corresponding to merchants, and an edge from a merchant A to merchant B with weight rAB corresponding to an agreement via which customers can convert 1 point issued by merchant B into rAB points issued by merchant A. We henceforth refer to A as the source node and B as the sink node of this edge and points issued by A as A-points and points issued by B as B-points. We aim to understand the structure of these coalition programs. We do so via studying a strategic network formation game. Our model incorporates the following critical aspects of these networks:

Approximately optimal scheduling of an M/G/1 queue with heavy tails

Distributions with a heavy tail are difficult to estimate. If the design of an optimal scheduling policy is sensitive to the details of heavy tail distributions of the service times, an approximately optimal solution is difficult to obtain. This paper shows that the mean optimal scheduling of an M/G/1 queue with heavy tailed service times does not present this difficulty and that an approximately optimal strategy can be derived by truncating the distributions.

Atomic hierarchical routing games in communication networks

Theoretical studies on routing games in networks have so far dealt with reciprocal congestion effects between routing entities. But, with the advent of technologies like Cognitive Radio, we have networks which support differentiation of flows. In a two priority level model a user can be high priority or low priority and there is a cost for such a classification. The point of departure of this model from the traditional routing scenarios is the absence of reciprocity in the congestion effects: The low priority flow faces congestion from both high priority as well as low priority flow while the high priority flow is immune to the congestion effects from the low priority flow. This hierarchy is naturally present in contexts where there are primary (licensed) users and secondary (unlicensed) users who can sense their environment because there are equipped with a cognitive radio. We study such kind of routing scenarios for the cases of atomic users. We establish the existence and the uniqueness of Nash equilibrium and further we show the existence of a potential function for linear congestion costs and a certain priority classification pricing scheme. Natural applications of this model to Cognitive Radio are also pointed out.