Peter Jacko

A nearly-optimal index rule for scheduling of users with abandonment

We analyze a comprehensive model for multi-class job scheduling accounting for user abandonment, with the objective of minimizing the total discounted or time-average sum of linear holding costs and abandonment penalties. We assume geometric service times and Bernoulli abandonment probabilities. We solve analytically the case in which there are 1 or 2 users in the system to obtain an optimal index rule. For the case with more users we use recent advances from the restless bandits literature to obtain a new simple index rule, denoted by AJN, which we propose to use also in the system with arrivals. In the problem without abandonment, the proposed rule recovers the cµ-rule which is well-known to be optimal both without and with arrivals. Under certain conditions, our rule is equivalent to the cµ/θ-rule, which was recently proposed and shown to be asymptotically optimal in a multi-server system with overload conditions. We present results of an extensive computational study that suggest that our rule is almost always superior or equivalent to other rules proposed in the literature, and is often optimal.

Optimal anticipative congestion control of flows with time-varying input stream

This paper is concerned with a new type of congestion control method that we call anticipative congestion control, which exploits probabilistic information available at a network node about congestion at other nodes. Motivated by the Internet flows behaving according to the Transmission Control Protocol, we consider a flow with time-varying input stream. We design a Markov decision process model for flow admission control and characterize the Whittle index in a closed form. This index measures the efficiency of flow data transmission at a router. We prove that such an index policy is optimal and that it further implies optimality of threshold policies. We apply the results to obtain an expression of the index for a single-bottleneck flow under several types of fairness criteria.

Congestion control of TCP flows in Internet routers by means of index policy

In this paper we address the problem of fast and fair transmission of flows in a router, which is a fundamental issue in networks like the Internet. We model the interaction between a source using the Transmission Control Protocol (TCP) and a bottleneck router with the objective of designing optimal packet admission controls in the router queue. We focus on the relaxed version of the problem obtained by relaxing the fixed buffer capacity constraint that must be satisfied at all time epoch. The relaxation allows us to reduce the multi-flow problem into a family of single-flow problems, for which we can analyze both theoretically and numerically the existence of optimal control policies of special structure. In particular, we show that for a variety of parameters, TCP flows can be optimally controlled in routers by so-called index policies, but not always by threshold policies. We have also implemented the index policy in Network Simulator-3 and tested in a simple topology their applicability in real networks. The simulation results show that the index policy achieves a wide range of desirable properties with respect to fairness between different TCP versions, across users with different round-trip-time and minimum buffer required to achieve full utility of the queue.

Scheduling of users with markovian time-varying transmission rates

We address the problem of developing a well-performing and implementable scheduler of users with wireless connection to the base station. The main feature of such real-life systems is that the quality conditions of the user channels are time-varying, which turn into the time-varying transmission rate due to different modulation and coding schemes. We assume that this phenomenon follows a Markovian law and most of the discussion is dedicated to the case of three quality conditions of each user, for which we characterize an optimal index policy and show that threshold policies (of giving higher priority to users with higher transmission rate) are not necessarily optimal. For the general case of arbitrary number of quality conditions we design a scheduler and propose its two practical approximations, and illustrate the performance of the proposed index-based schedulers and existing alternatives in a variety of simulation scenarios.

Modeling D2D communications with LTE and WiFi

In this work we propose a roadmap towards the analytical understanding of Device-to-Device (D2D) communications in LTE-A networks. Various D2D solutions have been proposed, which include inband and outband D2D transmission modes, each of which exhibits different pros and cons in terms of complexity, interference, and spectral efficiency achieved. We go beyond traditional mode optimization and mode-selection schemes. Specifically, we formulate a general problem for the joint per-user mode selection, connection activation and resource scheduling of connections using both LTE and WiFi resources.

Opportunistic scheduling of flows with general size distribution in wireless time-varying channels

In this paper we study how to design an opportunistic scheduler when flow sizes have a general service time distribution with the objective of minimizing the expected holding cost. We allow the channel condition to have two states which in particular covers the important special case of ON/OFF channels. We formulate the problem as a multi-armed restless bandit problem, a particular class of Markov decision processes. Since an exact solution is out of reach, we characterize in closed-form the Whittle index, which allows us to define a heuristic scheduling rule for the problem. We then particularize the index to the important subclass of distributions with a decreasing hazard rate. We finally evaluate the performance of the proposed Whittle-index based scheduler by simulation of a wireless network. The numerical results show that the performance of the proposed scheduler is very satisfactory.

An opportunistic and non-anticipating size-aware scheduling proposal for mean holding cost minimization in time-varying channels

In this paper we study how to design a scheduling strategy aimed at minimizing the average holding cost for flows with general size distribution when the feasible transmission rate of each user varies randomly over time. We employ a Whittle-index-based approach in order to achieve an opportunistic and non-anticipating size-aware scheduling index rule proposal. When the flow size distribution belongs to the Decreasing Hazard Rate class, we propose the so-called Attained Service Potential Improvement index rule, which consists in giving priority to the flows with the highest ratio between the current attained-service-dependent completion probability and the expected potential improvement of this completion probability. We further analyze the performance of the proposed scheduler, concluding that it outperforms well-known opportunistic disciplines.

A Bayesian adaptive design for clinical trials in rare diseases

Development of treatments for rare diseases is challenging due to the limited number of patients available for participation. Learning about treatment effectiveness with a view to treat patients in the larger outside population, as in the traditional fixed randomised design, may not be a plausible goal. An alternative goal is to treat the patients within the trial as effectively as possible. Using the framework of finite-horizon Markov decision processes and dynamic programming (DP), a novel randomised response-adaptive design is proposed which maximises the total number of patient successes in the trial and penalises if a minimum number of patients are not recruited to each treatment arm. Several performance measures of the proposed design are evaluated and compared to alternative designs through extensive simulation studies using a recently published trial as motivation. For simplicity, a two-armed trial with binary endpoints and immediate responses is considered. Simulation results for the proposed design show that: (i) the percentage of patients allocated to the superior arm is much higher than in the traditional fixed randomised design; (ii) relative to the optimal DP design, the power is largely improved upon and (iii) it exhibits only a very small bias and mean squared error of the treatment effect estimator. Furthermore, this design is fully randomised which is an advantage from a practical point of view because it protects the trial against various sources of bias. As such, the proposed design addresses some of the key issues that have been suggested as preventing so-called bandit models from being implemented in clinical practice.

Resource capacity allocation to stochastic dynamic competitors: knapsack problem for perishable items and index-knapsack heuristic

In this paper we propose an approach for solving problems of optimal resource capacity allocation to a collection of stochastic dynamic competitors. In particular, we introduce the knapsack problem for perishable items, which concerns the optimal dynamic allocation of a limited knapsack to a collection of perishable or non-perishable items. We formulate the problem in the framework of Markov decision processes, we relax and decompose it, and we design a novel index-knapsack heuristic which generalizes the index rule and it is optimal in some specific instances. Such a heuristic bridges the gap between static/deterministic optimization and dynamic/stochastic optimization by stressing the connection between the classic knapsack problem and dynamic resource allocation. The performance of the proposed heuristic is evaluated in a systematic computational study, showing an exceptional near-optimality and a significant superiority over the index rule and over the benchmark earlier-deadline-first policy. Finally we extend our results to several related revenue management problems.

Resource-sharing in a single server with time-varying capacity

We investigate the problem of sharing the resources of a single server with time-varying capacity with the objective of minimizing the mean delay. We formulate the resource allocation problem as a Markov Decision Process. The problem is not solvable analytically in full generality, and we thus set out to obtain an approximate solution. In our main contribution, we extend the framework of multi-armed bandits to develop a heuristic solution of index type. At every given time, the heuristic assigns an index to every user that depends solely on its current state, and serves the user with highest current index value. We show that in the case of constant capacity, the heuristic policy is equivalent to the so-called Gittins index rule, which is known to be optimal under the assumption of constant capacity.