Thomas P. Hayes

The adwords problem: online keyword matching with budgeted bidders under random permutations

We consider the problem of a search engine trying to assign a sequence of search keywords to a set of competing bidders, each with a daily spending limit. The goal is to maximize the revenue generated by these keyword sales, bearing in mind that, as some bidders may eventually exceed their budget, not all keywords should be sold to the highest bidder. We assume that the sequence of keywords (or equivalently, of bids) is revealed on-line. Our concern will be the competitive ratio for this problem versus the off-line optimum. We extend the current literature on this problem by considering the setting where the keywords arrive in a random order. In this setting we are able to achieve a competitive ratio of 1-ε under some mild, but necessary, assumptions. In contrast, it is already known that when the keywords arrive in an adversarial order, the best competitive ratio is bounded away from 1. Our algorithm is motivated by PAC learning, and proceeds in two parts: a training phase, and an exploitation phase.

Error limiting reductions between classification tasks

We introduce a reduction-based model for analyzing supervised learning tasks. We use this model to devise a new reduction from multi-class cost-sensitive classification to binary classification with the following guarantee: If the learned binary classifier has error rate at most ε then the cost-sensitive classifier has cost at most 2ε times the expected sum of costs of all possible lables. Since cost-sensitive classification can embed any bounded loss finite choice supervised learning task, this result shows that any such task can be solved using a binary classification oracle. Finally, we present experimental results showing that our new reduction outperforms existing algorithms for multi-class cost-sensitive learning.

A non-Markovian coupling for randomly sampling colorings

We study a simple Markov chain, known as the Glauber dynamics, for randomly sampling (proper) k-colorings of an input graph G on n vertices with maximum degree /spl Delta/ and girth g. We prove the Glauber dynamics is close to the uniform distribution after O(n log n) steps whenever k > (1 + /spl epsiv/)/spl Delta/, for all /spl epsiv/ > 0, assuming g /spl ges/ 9 and /spl Delta/ = /spl Omega/(log n). The best previously known bounds were k > 11/spl Delta//6 for general graphs, and k > 1.489/spl Delta/ for graphs satisfying girth and maximum degree requirements. Our proof relies on the construction and analysis of a non-Markovian coupling. This appears to be the first application of a non-Markovian coupling to substantially improve upon known results.

Randomly coloring graphs of girth at least five

We improve rapid mixing results for the simple Glauber dynamics designed to generate a random k-coloring of a bounded-degree graph.Let G be a graph with maximum degree Δ = Ω(log n), and girth ≥ 5. We prove that if k > Α Δ, where Α ≈ 1.763 then Glauber dynamics has mixing time O(n log n). If girth(G) ≥ 6 and k > Β Δ, where Β ≈ 1.489 then Glauber dynamics has mixing time O(n log n). This improves a recent result of Molloy, who proved the same conclusion under the stronger assumptions that Δ=Ω(log n) and girth Ω(log Δ). Our work suggests that rapid mixing results for high girth and degree graphs may extend to general graphs.Analogous results hold for random graphs of average degree up to n¼, compared with polylog(n), which was the best previously known.Some of our proofs rely on a new Chernoff-Hoeffding type bound, which only requires the random variables to be well-behaved with high probability. This tail inequality may be of independent interest.

Coupling with the stationary distribution and improved sampling for colorings and independent sets

We present an improved coupling technique for analyzing the mixing time of Markov chains. Using our technique, we simplify and extend previous results for sampling colorings and independent sets. Our approach uses properties of the stationary distribution to avoid worst-case configurations which arise in the traditional approach.As an application, we show that for <i>k</i>/Δ > 1.764, the Glauber dynamics on <i>k</i>-colorings of a graph on <i>n</i> vertices with maximum degree Δ converges in <i>O</i>(<i>n</i> log <i>n</i>) steps, assuming Δ = Ω(log <i>n</i>) and that the graph is triangle-free. Previously, girth ≥ 5 was needed.As a second application, we give a polynomial-time algorithm for sampling weighted independent sets from the Gibbs distribution of the hard-core lattice gas model at fugacity λ < (1 - ε)<i>e</i>/Δ, on a regular graph <i>G</i> on <i>n</i> vertices of degree Δ = Ω(log <i>n</i>) and girth ≥ 6. The best known algorithm for general graphs currently assumes λ < 2/(Δ - 2).

The forgiving tree: a self-healing distributed data structure

We consider the problem of self-healing in peer-to-peer networks that are under repeated attack by an omniscient adversary. We assume that the following process continues for up to n rounds where n is the total number of nodes initially in the network: the adversary deletesan arbitrary node from the network, then the network responds by quickly adding a small number of new edges. We present a distributed data structure that ensures two key properties. First, the diameter of the network is never more than O(log Delta) times its original diameter, where Delta is the maximum degree of the network initially. We note that for many peer-to-peer systems, Delta is polylogarithmic, so the diameter increase would be a O(loglog n) multiplicative factor. Second, the degree of any node never increases by more than 3 over its original degree. Our data structure is fully distributed, has O(1) latency per round and requires each node to send and receive O(1) messages per round. The data structure requires an initial setup phase that has latency equal to the diameter of the original network, and requires, with high probability, each node v to send O(log n) messages along every edge incident to v. Our approach is orthogonal and complementary to traditional topology-based approaches to defending against attack.

Membrane structure of OsO4-fixed erythrocytes viewed "face on" by electron microscope techniques.

Abstract A carbon replica technique is used to study the membrane of OsO 4 -fixed rat erythrocytes. When metal-shadowed, the replicas show pebbly or granular structure at the level of 400 to 500 A. Examination of unshadowed replicas reveals organic material, attached to the carbon, in what seems to be a repeatedly looped, filamentous structure. The possible relationship of these filaments to elinin is discussed.

The cost of the missing bit: communication complexity with help

We generalize the multiparty communication model of Chandra, Furst, and Lipton (1983) to functions with b-bit output (b = 1 in the CFL model). We allow the players to receive up to b − 1 bits of information from an all-powerful benevolent Helper who can see all the input. Extending results of Babai, Nisan, and Szegedy (1992) to this model, we construct families of explicit functions for which Ω(n/ck) bits of communication are required to find the “missing bit,” where n is the length of each player’s input and k is the number of players. As a consequence we settle the problem of separating the one-way vs. multiround communication complexities (in the CFL sense) for k ≤ (1 − ) log n players, extending a result of Nisan and Wigderson (1991) who demonstrated this separation for k = 3 players. As a byproduct we obtain Ω(n/ck) lower bounds for the multiparty complexity (in the CFL sense) of new families of explicit boolean functions (not derivable from BNS). ∗Department of Computer Science, University of Chicago, 1100 East 58th Street, Chicago, IL 60637. e-mail: laci@cs.uchicago.edu. Supported in part by NSA grant MSPR-96G-184 †Department of Mathematics, University of Chicago, 5734 S. University Avenue, Chicago, IL 60637. e-mail: hayest@math.uchicago.edu. Supported in part by an NSF Graduate Fellowship. ‡Department of Computer Science, Northeastern Illinois University, 5500 N. St. Louis Avenue, Chicago, IL 60625-4699. e-mail: pgkimmel@neiu.edu

Convergence of MCMC and Loopy BP in the Tree Uniqueness Region for the Hard-Core Model

We study the hard-core (gas) model defined on independent sets of an input graph where the independent sets are weighted by a parameter (aka fugacity) λ > 0. For constant Δ, previous work of Weitz (2006) established an FPTAS for the partition function for graphs of maximum degree Δ when λ <; λ_c(Δ). Sly (2010) showed that there is no FPRAS, unless NP=RP, when λ > λ_c(Δ). The threshold λ_c(Δ) is the critical point for the statistical physics phase transition for uniqueness/non-uniqueness on the infinite Δ-regular tree. The running time of Weitzs algorithm is exponential in log Δ. Here we present an FPRAS for the partition function whose running time is O* (n²). We analyze the simple single-site Markov chain known as the Glauber dynamics for sampling from the associated Gibbs distribution. We prove there exists a constant Δ₀ such that for all graphs with maximum degree Δ > Δ₀ and girth > 7 (i.e., no cycles of length ≤ 6), the mixing time of the Glauber dynamics is O(nlog n) when λ <; λ_c(Δ). Our work complements that of Weitz which applies for small constant Δ whereas our work applies for all Δ at least a sufficiently large constant Δ₀ (this includes Δ depending on n = IVI). Our proof utilizes loopy BP (belief propagation) which is a widely-used algorithm for inference in graphical models. A novel aspect of our work is using the principal eigenvector for the BP operator to design a distance function which contracts in expectation for pairs of states that behave like the BP fixed point. We also prove that the Glauber dynamics behaves locally like loopy BP. As a byproduct we obtain that the Glauber dynamics, after a short burn-in period, converges close to the BP fixed point, and this implies that the fixed point of loopy BP is a close approximation to the Gibbs distribution. Using these connections we establish that loopy BP quickly converges to the Gibbs distribution when the girth ≥ 6 and λ <; λ_c(Δ).

Liftings of tree-structured Markov chains

A “lifting” of a Markov chain is a larger chain obtained by replacing each state of the original chain by a set of states, with transition probabilities defined in such a way that the lifted chain projects down exactly to the original one. It is well known that lifting can potentially speed up the mixing time substantially. Essentially all known examples of efficiently implementable liftings have required a high degree of symmetry in the original chain. Addressing an open question of Chen, Lovasz and Pak, we present the first example of a successful lifting for a complex Markov chain that has been used in sampling algorithms. This chain, first introduced by Sinclair and Jerrum, samples a leaf uniformly at random in a large tree, given approximate information about the number of leaves in any subtree, and has applications to the theory of approximate counting and to importance sampling in Statistics. Our lifted version of the chain (which, unlike the original one, is non-reversible) gives a significant speedup over the original version whenever the error in the leaf counting estimates is o(1). Our lifting construction, based on flows, is systematic, and we conjecture that it may be applicable to other Markov chains used in sampling algorithms.