Bounded-Loss Private Prediction Markets
aa r X i v : . [ c s . G T ] O c t Bounded-Loss Private Prediction Markets
Rafael Frongillo
Colorado Boulder [email protected]
Bo Waggoner
Microsoft Research [email protected]
Abstract
Prior work has investigated variations of prediction markets that preserve partici-pants’ (differential) privacy, which formed the basis of useful mechanisms for pur-chasing data for machine learning objectives. Such markets required potentiallyunlimited financial subsidy, however, making them impractical. In this work, wedesign an adaptively-growing prediction market with a bounded financial subsidy,while achieving privacy, incentives to produce accurate predictions, and precisionin the sense that market prices are not heavily impacted by the added privacy-preserving noise. We briefly discuss how our mechanism can extend to the data-purchasing setting, and its relationship to traditional learning algorithms.
In a prediction market, a platform maintains a prediction (usually a probability distribution or anexpectation) of a future random variable such as an election outcome. Participants’ trades of finan-cial securities tied to this event are translated into updates to the prediction. Prediction markets,designed to aggregate information from participants, have gained a substantial following in the ma-chine learning literature. One reason is the overlap in goals (predicting future outcomes) as well astechniques (convex analysis, Bregman divergences), even at a deep level: the form of market updatesin standard automated market makers have been shown to mimic standard online learning or opti-mization algorithms in many settings [2, 3, 11, 12]. Beyond this research-level bridge, recent papershave suggested prediction market mechanisms as a way of crowdsourcing data or algorithms formachine learning, usually by providing incentives for participants to repeatedly update a centralizedhypothesis or prediction [4, 15].One recently-proposed mechanism to purchase data or hypotheses from participants is that of Wag-goner, et al. [15], in which participants submit updates to a centralized market maker, either bydirectly altering the hypothesis, or in the form of submitted data; both are interpreted as buying orselling shares in a market, paying off according to a set of holdout data that is revealed after the closeof the market. The authors then show how to preserve differential privacy for participants, mean-ing that the content of any individual update is protected, as well as natural accuracy and incentiveguarantees.One important drawback of Waggoner, et al. [15], however, is the lack of a bounded worst-case loss guarantee: as the number of participants grows, the possible financial liability of the mechanismgrows without bound. In fact, their mechanism cannot achieve a bounded worst-case loss withoutgiving up privacy guarantees. Subsequently, Cummings, et al. [8] show that all differentially-privateprediction markets of the form proposed in [15] must suffer from unbounded financial loss in theworst case. Intuitively, one could interpret this negative result as saying that the randomness ofthe mechanism, which must be introduced to preserve privacy, also creates arbitrage opportunitiesfor participants: by betting against the noise, they collectively expect to make an unbounded profitfrom the market maker. Nevertheless, Cummings, et al. leave open the possibility that mechanismsoutside the mold of Waggoner, et al. could achieve both privacy and a bounded worst-case loss. n this paper, we give such a mechanism: the first private prediction market framework with abounded worst-case loss. When applied to the crowdsourcing problems stated above, this now allowsthe mechanism designer to maintain a fixed budget. Our construction and proof proceeds in twosteps.We first show that by adding a small transaction fee to the mechanism of [15], one can eliminatefinancial loss due to arbitrage while maintaining the other desirable properties of the market. Thekey idea is that a carefully-chosen transaction fee can make each trader subsidize (in expectation)any arbitrage that may result from the noise preserving her privacy. Unless prices already match herbeliefs quite closely, however, she still expects to make a profit by paying the fee and participating.We view this as a positive result both conceptually—it shows that arbitrage opportunities are notan insurmountable obstacle to private markets—and technically—the designer budget grows veryslowly, only O ((log T ) ) , with the number of participants T .Nonetheless, this first mechanism is still not completely satisfactory, as the budget is superconstantin T , and T must be known in advance. This difficulty arises not from arbitrage, but (apparently) adeeper constraint imposed by privacy that forces the market to be large relative to the participants.Our second and main result overcomes this final hurdle. We construct a sequence of adaptively-growing markets that are syntactically similar to the “doubling trick” in online learning. The keyidea is that, in the market from our first result, only about (log T ) of the T participants can be informational traders; after this point, additional participants do not cost the designer any morebudget, yet their transaction fees can raise significant funds. So if the end of a stage is reached, themarket activity has actually generated a surplus which subsidizes the initial portion of the next stageof the market. In a cost-function based prediction market, there is an observable future outcome Z taking valuesin a set Z . The goal is to predict the expectation of a random variable φ : Z → R d . We assume φ is a bounded random variable, as otherwise prediction markets with bounded financial loss are notpossible. Participants will buy from the market contracts , each parameterized by a vector r ∈ R d .The contract represents a promise for the market to pay the owner r · φ ( Z ) when Z is observed.Adopting standard financial terminology, in our model there are d securities j = 1 , . . . , d , and theowner of a share in security j will receive a payoff of φ ( Z ) j , that is, the j th component of the randomvariable. Thus a contract r ∈ R d contains r j shares of security j and pays off P dj =1 r j φ ( Z ) j = r · φ ( Z ) . Note that r j < , or “short selling” security j , is allowed.The market maintains a market state q t ∈ R d at time t = 0 , . . . , T , with q = 0 . Each trader t = 1 , . . . , T arrives sequentially and purchases a contract dq t ∈ R d , and the market state is updatedto q t = q t − + dq t . In other words, q t = P ts =1 dq s , the sum of all contracts purchased up to time t .The price paid by each participant is determined by a convex cost function C : R d → R . Intuitively, C maps q t to the total price paid by all agents so far, C ( q t ) . Thus, participant t making trade dq t when the current state is q t − pays C ( q t − + dq t ) − C ( q t − ) . Notice that the instantaneous prices p t = ∇ C ( q t ) represent the current price per unit of infinitesimal purchases, with the j th coordinaterepresenting the current price per share of the j th security.The prices ∇ C ( q ) are interpreted as predictions of E φ ( Z ) , as an agent who believes the j th co-ordinate is too low will purchase shares in it, raising its price, and so on. This can be formalizedthrough a learning lens: It is known [2] that agents in such a market maximize expected profit byminimizing an expected Bregman divergence between φ ( Z ) and ∇ C ( q ) ; of course, it is known that ∇ C ( q ) = E φ ( Z ) minimizes risk for any divergence-based loss [1, 6, 13]. (The Bregman divergenceis that corresponding to C ∗ , the convex conjugate of C .) Price Sensitivity.
The price sensitivity of a cost function C is a measure of how quickly pricesrespond to trades, similar to “liquidity” discussed in Abernethy et al. [2, 5] and earlier works. For-mally, the price sensitivity λ of C is the supremum of the operator norm of the Hessian of C , withrespect to the ℓ norm. In other words, if c = k q − q ′ k shares are purchased, then the change inprices k∇ C ( q ) − ∇ C ( q ′ ) k is at most λc . For convenience we will assume C is twice differentiable, though this is not necessary. S , the loss can be bounded by(a constant times) the largest possible score. Hence, scaling S by a factor λ immediately scales theloss bound by λ as well. Recall that S is defined by a convex function G , the convex conjugateof C . Scaling S by λ is equivalent to scaling G by λ . By standard results in convex analysis,this is equivalent to transforming C into C λ ( q ) = λ C ( λq ) , an operation known as the perspectivetransform. This in turn scales the price sensitivity by λ by the properties of the Hessian.Price sensitivity is also related to the total number of trades required to change the prices in a market.If we assume each trade consists of at most one share in each security, then λ trades are necessaryto shift the predictions to an arbitrary point from an arbitrary point. Convention: normalized, scaled C . In the remainder of the paper, we will suppose that we startwith some convex cost function C whose price sensitivity equals and worst-case loss bounded bysome constant B . Then, to obtain price sensitivity λ , we use the cost function C ( · ) = λ C ( λ · ) . Asdiscussed above, C has price sensitivity at most λ and a worst-case loss bound of B = B /λ . (Thisassumption is without loss of generality, as any cost function that guarantees a bounded worst-caseloss can be scaled such that its price sensitivity is .) To achieve differential privacy for trades of a bounded size (which will be assumed), the generalapproach is to add random noise to the “true” market state q and publish this noisy state ˆ q . Theprivacy level thus determines how close ˆ q is to q . The distance from ∇ C (ˆ q ) to ∇ C ( q ) is thencontrolled by the price sensitivity λ . For a fixed noise and privacy level, a smaller λ leads to smallimpact of noise on prices, meaning very good accuracy. However, decreasing λ does not come forfree: the worst-case financial loss of to the market designer scales as /λ .The market of [15] adds controlled and correlated noise over time, in a manner similar to the “contin-ual observation” technique of differential privacy. This reduces the influence of noise on accuracy topolylogarithmic in T , the number of participants. Their main result for the prediction market settingstudied here is as follows. Theorem 1 ([15]) . Assuming that all trades satisfy k dq t k ≤ , the private mechanism is ǫ -differentially private in the trades dq , . . . , dq T with respect to the output ˆ q , . . . , ˆ q T . Further, tosatisfy k p t − ˆ p t k ≤ α for all t , except with probability γ , it suffices for the price sensitivity to be λ ∗ = α ǫ √ d ⌈ log T ⌉ ln(2 T d/γ ) . (1) This paper builds on the work of Waggoner et al. [15] to overcome the negative results of Cummingset al. [8]. Here, we formalize our setting and four desirable properties we hope to achieve.Write a prediction market mechanism as a function M taking inputs ~dq = dq , . . . , dq T andoutputting a sequence of market states ˆ q , . . . , ˆ q T . Here ˆ q t is thought of as a noisy version of q t = P s ≤ t dq s . Each of these states is associated with a prediction ˆ p t in the set of possible prices(expectations of φ ), while the state q t is associated with the “true” underlying prediction p t . Definition 1 (Privacy) . M satisfies ( ǫ, δ ) -differential privacy if for all pairs of inputs ~dq, ~dq ′ differ-ing by only a single participants’ entry, and for all sets S of possible outputs, Pr[ M ( ~dq ) ∈ S ] ≤ e ǫ Pr[ M ( ~dq ′ ) ∈ S ] + δ . If furthermore δ = 0 , we say M is ǫ -differentially private . Definition 2 (Precision) . M has ( α, γ ) precision if for all ~dq , with probability − γ , we have k ˆ p t − p t k ≤ α for all t . Definition 3 (Incentives) . M has β -incentive to participate if, for all beliefs p = E φ ( Z ) , if at anypoint k ˆ p t − p k ∞ > β , then there exists a participation opportunity that makes a strictly positiveprofit in expectation with respect to p .For the budget guarantee, we must formalize the notion that participants may respond tothe noise introduced by the mechanism. Following Cummings et al. [8], let a trader trategy ~s = ( s , . . . , s T ) where each s t is a possibly-randomized function of the form s t ( dq , . . . , dq t − ; ˆ q , . . . , ˆ q t − ) = dq t , i.e. a strategy taking the entire history prior to t and out-putting a trade dq t . Let L ( M, ~s, z ) be a random variable denoting the financial loss of the market M against trader strategy ~s when Z = z , which for the mechanism described above is simply L ( M, ~s, z ) = T X t =1 (cid:2) C (ˆ q t ) − C (ˆ q t + dq t ) − dq t · φ ( z ) (cid:3) . Definition 4. M guarantees designer budget B if, for any trader strategy ~s and all z , E L ( M, ~s, z ) ≤ B , where the expectation is over the randomness in M and each s t . The private market of Waggoner et al. [15] causes unbounded loss for the market maker in two ways.The first is from traders betting against the random noise introduced to protect privacy. This is a keyidea leveraged by Cummings et al. [8] to show negative results for private markets. In this section,we show that a transaction fee can be chosen to exactly balance the expected profit from this typeof arbitrage. We will show that this fee is still small enough to allow for very accurate prices. This transaction fee restores the worst-case loss guarantee to the inverse of the price sensitivity, justas in a non-private market. The second way the market causes unbounded loss is to require pricesensitivity to shrink as a function of T ; this is addressed in the next section.We show that with this carefully-chosen fee, the market still achieves precision, incentive, and pri-vacy guarantees, but now with a worst-case market maker loss of O ((log T ) ) , much improved overthe naïve O ( T ) bound. This is viewed as a positive result because the worst-case loss is growingquite slowly in the total number of participants, and moreover matches the fundamental “informa-tional” worst-case loss one expects with price sensitivity λ ∗ . Here we recall the private market mechanism of [15], adapted to the prediction market setting fol-lowing [8]. We will express the randomness of the mechanism in terms of a “noise trader” for bothintuition and technical convenience. The market is defined by a cost function C with price sensi-tivity λ , and parameters c (transaction fee), ǫ (privacy), α, γ (precision), and T (maximum numberof participants). There is a special trader we call the noise trader who is controlled by the designer.All actions of the noise trader are hidden and known only by the designer. The designer publishesan initial market state q = ˆ q = 0 . Let T ′ denote the actual number of arrivals, with T ′ ≤ T byassumption. Then, for t = 1 , . . . , T ′ :1. Participant t arrives, pays a fee of c , and purchases bundle dq t with k dq t k ≤ . Thepayment is C (ˆ q t + dq t ) − C (ˆ q t ) .2. The noise trader purchases a randomly-chosen bundle z t , called a noise trade, after sellingoff some subset { z t , . . . , z t k } of previously purchased noise trades for t i < t , accordingto a predetermined schedule described below. Letting w t = z t − P ki =1 z t i denote this netnoise bundle, the noise trader is thus charged C (ˆ q t + dq t + w t ) − C (ˆ q t + dq t ) .3. The “true” market state is updated to q t = q t − + dq t , but is not revealed.4. The noisy market state is updated to ˆ q t = ˆ q t − + dq t + w t and is published.Finally, z ∈ Z is observed and each participant t receives a payment dq t · φ ( z ) . For the sake ofbudget analysis, we suppose that at the close of the market, the noise trader sells back all of herremaining bundles; letting w T ′ be the sum of these bundles, she is charged C (ˆ q T ′ − w T ′ ) − C (ˆ q T ′ ) . Noise trades.
Each z t is a d -dimensional vector with each coordinate drawn from an independentLaplace distribution with parameter b = 2 ⌈ log T ⌉ /ǫ . To determine which bundles z s are sold attime t , write t = 2 j m where m is odd, and sell all bundles z s purchased during the previous Intuitively, it is enough for the fee to cover arbitrage amounts in expectation, because a trader must pay thefee to trade before the random noise is drawn and any arbitrage opportunity is revealed. For instance, if the current price of a security is . and a trader believes the true price should be . ,she will purchase a share if the fee is c < . . (For privacy, we limit each trade to a fixed size, say, one share.) j − time steps which are not yet sold. Thus, the noise trader will sell bundles purchased at times s = t − , t − , t − , t − , . . . , t − j − ; in particular, when t is odd we have j = 0 , so no previousbundles will be sold. Budget.
The total loss of the market designer can now be written as the sum of three terms: the lossof the market maker, the loss of the noise trader, and the gain from transaction fees. By convention,the noise trader eventually sells back all bundles it purchases and is left with no shares remaining. L ( M, ~s, z ) = net loss of market maker z }| { T ′ X t =1 C (ˆ q t − ) − C (ˆ q t − + dq t ) + dq t · φ ( z ) + net loss of noise trader z }| { T ′ X t =1 C (ˆ q t − + dq t ) − C (ˆ q t ) + fees z}|{ cT ′ . (2)The main result of this section is as follows. Theorem 2.
When each arriving participant pays a transaction fee c = α , the private market withany λ ≤ λ ∗ from eq. (1) satisfies ǫ -differential privacy, ( α, γ ) -precision, α -incentive to trade, andbudget bound B λ , where B is the budget bound of the underlying cost function C . The differential privacy and precision claims follow directly from the prior results, as nothing haschanged to impact them. The incentive claim is not technically involved, but perhaps subtle: thetransaction fee should be high enough to eliminate expected profit from arbitrage, yet low enoughto allow for profit from information. The key point is that the transaction fee is a constant, but thefarther the prices are from the trader’s belief, the more money she expects to make from a constant-sized trade. The transaction fee creates a ball of size α around the current prices where, if one’sbelief lies in that ball, then participation is not profitable.We give most of the proof of the designer budget bound, with some claims deferred to the fullversion. Lemma 1 (Budget bound) . The transaction-fee private market with any price sensitivity λ ≤ λ ∗ guarantees a designer budget bound of B λ .Proof. Let c be the transaction fee; we will later take c = α . Then the worst-case loss from eq. (2)is W C ( λ, T ′ ) := W C ( λ, T ′ ) + N T L ( λ, T ′ ) − T ′ c , where W C ( λ, T ′ ) is the worst-case loss of a standard prediction market maker with parameter λ and T ′ participants, N T L ( λ, T ′ ) is the worst-case noise trader loss, and T ′ c is the revenue from T ′ transaction fees of size c each.The worst-case loss of a standard prediction market maker is well-known; see e.g. [2]. By ournormalization and definition of price sensitivity, we thus have W C ( λ, T ′ ) ≤ B λ .To bound the noise trader loss N T L ( λ, T ′ ) , we will consider each bundle z t purchased by the noisetrader. The idea is to bound the difference in price between the purchase and sale of z t . For analysis,we suppose that at each t , the noise trader first sells any previous bundles (e.g. at t = 4 , first selling z and then selling z ), and then purchases z t .Now let b ( t ) be the largest power of that divides t . Let q t buy and q t sell be the market state just beforethe noise trader purchases z t and just after she sells z t , respectively. Claim 1.
For each t , exactly b ( t ) traders arrive between the purchase and the sale of bundle z t ;furthermore, q t sell − q t buy is exactly equal to the sum of these participants’ trades. For example, suppose t is odd. Then only one participant arrives between the purchase and sale of z t . Furthermore, z t is the last bundle purchased by the noise trader at time t and is the first sold attime t + 1 , so the difference in market state is exactly z t plus that participant’s trade. Claim 2.
If the noise trader purchases and later sells z t , then her net loss in expectation over z t (but for any trader behavior in response to z t ), is at most λb ( t ) K where K = E k z t k .
5e now sum over all bundles z t purchased by the noise trader, i.e. at time steps , . . . , T ′ . Recallthat the noise trader sells back every bundle z t she purchases. Thus, her total payoff is the sumover t of the difference in price at which she buys z t and price at which she sells it. For each j = 0 , . . . , log T ′ − , there are j different steps t with b ( t ) = T ′ / j +1 . The total loss is thus, N T L ( λ, T ′ ) ≤ log T ′ − X j =0 j T ′ j +1 λK = T ′ log T ′ λK . (3)Note that if the noise trader has some noise bundles left over after the final participant, we supposeshe immediately sells all remaining bundles back to the market maker in reverse order of purchase.Putting eq. (3) together with the above bound on W C gives W C ( λ, T ′ ) ≤ W C ( λ, T ′ ) + T ′ log T ′ λK − T ′ c ≤ B λ + T ′ ( K log T ′ λ − c ) , (4)which is in turn at most B /λ if we choose λ and the transaction fee c such that c ≥ K log T λ . Inother words, we take λ ≤ c/K log T .Finally, we can bound K = E k z t k from Claim 2 as follows: for each t , the components of the d -dimensional vector z t are each independent Lap( b ) variables with b = 2 ⌈ log T ⌉ /ǫ . By concavityof √· , we have K = E vuut d X i =1 z t ( i ) ≤ sX i E z t ( i ) = p d Var(Lap( b )) = √ db = 2 √ d ⌈ log T ⌉ ǫ . Therefore, it suffices to pick λ ≤ c ǫ √ d ⌈ log T ⌉ log T .
For c = α , this is in fact accomplished by the private, accurate market choosing λ ≤ λ ∗ fromeq. eq:lambda-star. Limitations of this result.
Unfortunately, Theorem 2 does not completely solve our problem: theother way that privacy impacts the market’s loss is by lowering the necessary price sensitivity to λ ∗ ≈ T ) as mentioned above, leading to a worst-case loss growing with T . It does not seempossible to address this via a larger transaction fee without giving up incentive to participate: tradersparticipate as long as their expected profit exceeds the fee, and collectively Ω(1 /λ ) of them canarrive making consistent trades all moving the prices in the same (correct) direction, so the totalpayout will still be Ω(1 /λ ) . In this section, we achieve our original goal by constructing an adaptively-growing prediction marketin which each stage, if completed, subsidizes the initial portion of the next.The market design is the following, with each T ( k ) to be chosen later. We run the transaction-fee pri-vate market above with T = T (1) , transaction fee α , and price sensitivity λ (1) = λ ∗ ( T (1) , α/ , γ/ from eq. (1). When (and if) T (1) participants have arrived, we create a new market whose initialstate is such that its prices match the final (noisy) prices of the previous one. We set T (2) and pricesensitivity λ (2) = λ ∗ ( T (2) , α/ , γ/ for the new market. We repeat, halving α and γ at each stageand increasing T in a manner to be specified shortly, until no more participants arrive. Theorem 3.
For any α, γ, ǫ , the adaptive market satisfies ǫ -differential privacy, α -incentive totrade, ( α, γ ) -accuracy, and a designer budget bound of B ≤ B √ dα ǫ (cid:18) ln 4608 B √ d γα ǫ (cid:19) , where B is the budget bound of the underlying unscaled cost function C . roof idea. We set T (1) = Θ (cid:0) B d ln( B d/γαǫ ) α ǫ (cid:1) , and T ( k ) = 4 T ( k − thereafter. The key willbe the following observation. The total “informational” profit available to the traders (by correctingthe initial market prices) is bounded by O (1 /λ ) , so if each trader expects to profit more than thetransaction fee c , then only O (1 /λc ) traders can all arrive and simultaneously profit. Indeed, ifall T participants arrive, then the total profit from transaction fees is Θ( T ) while the worst-case lossfrom the market is O (cid:0) (log T ) (cid:1) .We can leverage this observation to achieve a bounded worst-case loss with an “adaptive-liquidity”approach, similar in spirit to Abernethy et al. [5] but more technically similar to the doubling trickin online learning. Begin by setting λ (1) on the order of / (log T (1) ) = Θ(1) , and run a privatemarket for T (1) participants. If fewer than T (1) participants show up, the worst-case loss is order /λ (1) , a constant. If all T (1) participants arrive, then (for the right choice of constants) the markethas actually turned a profit Ω( T (1) ) from the transaction fees. Now set up a private market for T (2) = 4 T (1) traders with λ (2) on the order of / (log T (2) ) . If fewer than T (2) participants arrive,the worst-case loss is order /λ (2) . However, we will have chosen T (2) such that this loss is smallerthan the Ω( T (1) ) profit from the previous market. Hence, the total worst-case loss remains boundedby a constant.If all T (2) participants arrive, then again this market has turned a profit, which can be used tocompletely offset the worst-case loss of the next market, and so on. Some complications arise, as toachieve ( α, γ ) -precision, we must set α (1) , γ (1) , α (2) , γ (2) , . . . as a convergent series summing to α and γ ; and we must show that all of these scalings are possible in such a way that the transactionfees cover the cost of the next iteration. (An interesting direction for future work would be to replacethe iterative approach here with the continuous liquidity adaptation of [5].)More specifically, we prove that the loss in any round k that is not completed (not all participantsarrive) is at most α T ( k ) ; moreover, the profit in any round k that is completed is at least α T ( k ) .Of course, only one round is not completed: the final round k . If k = 1 , then the financial loss isbounded by λ (1) , a constant depending only on α, γ, ǫ . Otherwise, the total loss is the sum of thelosses across rounds, but the mechanism makes a profit in every round but k . Moreover, the loss inround k is at most α T ( k ) = α T ( k − , which is at most half of the profit in round k − . So if k ≥ ,the mechanism actually turns a net profit.While this result may seem paradoxical, note that the basic phenomenon appears in a classical (non-private) prediction market with a transaction fee, although to our knowledge this observation hasnot yet appeared in the literature. Specifically, a classical prediction market with budget bound B ,trades of size , and a small transaction fee α , will still have an α -incentive to participate, and theworst case loss will still be Θ( B ) ; this loss, however, can be extracted by as few as Θ(1) participants.Any additional participants must be in a sense disagreeing about the correct prices; their transactionfees go toward market maker profit, but they do not contribute further to worst-case loss.
While preserving privacy in prediction markets is well-motivated in the classical prediction mar-ket setting, it is arguably even more important in a setting where machine-learning hypotheses arelearned from private personal data. Waggoner et al. [15] develop mechanisms for such a settingbased on prediction markets, and further show how to preserve differential privacy of the partici-pants. Yet their mechanisms are not practical in the sense that the financial loss of the mechanismcould grow without bound. In this section, we sketch how our bounded-financial-loss market canalso be extended to this setting. This yields a mechanism for purchasing data for machine learningthat satisfies ǫ -differential privacy, α -precision and incentive to participate, and bounded designerbudget.To develop a mechanism which could be said to “purchase data” from participants, Waggoner etal. [15] extend the classical setting in two ways. The first is to make the market conditional , wherewe let Z = X × Y , and have independent markets C x : R d → R for each x . Trades in each markettake the form q x ∈ R d , which pay out q x · φ ( y ) upon outcome ( x ′ , y ) if x = x ′ , and zero if x = x ′ .Importantly, upon outcome ( x, y ) , only the costs associated to trades in the C x market are tallied.7he second is to change the bidding language using a kernel , a positive semidefinite function k : Z × Z → R . Here we think of contracts as functions f : Z → R in the reproducing kernel Hilbertspace (RKHS) F given by k , with basis { f z ( · ) = k ( z, · ) : z ∈ Z} . For example, we recover theconditional market setting with independent markets with the kernel k (( x, y ) , ( x ′ , y ′ )) = { x = x ′ } φ ( y ) · φ ( y ′ ) . The RKHS structure is natural here because a basis contract f z pays off at each z ′ according to the “covariance” structure of the kernel, i.e. the payoff of contract f z when z ′ occursequals f z ( z ′ ) = k ( z, z ′ ) . For example, when Y = {− , } one recovers radial basis classificationusing k (( x, y ) , ( x ′ , y ′ )) = yy ′ e − ( x − x ′ ) .These two modifications to classical prediction markets, given as Mechanism 2 in [15], have clearadvantages as a mechanism to “buy data”. One may imagine that each agent, arriving at time t ∈{ , . . . , T } , holds a data point ( x t , y t ) ∈ Z = X × Y . A natural purchase for this agent would be abasis contract f ( x t ,y t ) , as this corresponds to a payoff that is highest when the test data point actuallyequals ( x t , y t ) and decreases with distance as measured by the kernel structure.The importance of privacy now becomes even more apparent, as the data point ( x t , y t ) could beinformation sensitive to trader t . Fortunately, we can extend our main results to this setting. Todemonstrate the idea, we give a sketch of the result and proof below. Theorem 4 (Informal) . Let Z = X × Y where X is a compact subset of a finite-dimensional realvector space and Y is finite, and let positive semidefinite kernel k : Z × Z → R be given. Forany choices of accuracy parameters α, γ , privacy parameters ǫ, δ , trade size ∆ , and query limit Q , the kernel adaptive market satisfies ( ǫ, δ ) -differential privacy, ( α, γ ) -precision, α -incentive toparticipate, and a bounded designer budget.Proof Sketch. The precision property, i.e. that prices are approximately accurate despite privacy-preserving noise, follows from [15, Theorem 2], and the technique in Theorem 3 to combine theaccuracy and privacy of multiple epochs. The incentive to trade property is essentially unchanged,as a participants’ profit is still the improvement in expected Bregman divergence, which exceeds thetransaction fee unless prices are already accurate. It thus remains only to show a bounded designerbudget, which is slightly more involved. Briefly, Claim 1 goes through unchanged, and Claim 2holds as written where now C becomes C x and z t becomes z t ( x ) = f t ( x, · ) , i.e., the trade at time t restricted to the C x market alone.The remainder of Lemma 1 now proceeds with one modification regarding the constant K . In eq. (3),the expression for the noise trader loss becomes N T L ( λ, T ′ ) = E (cid:2) sup x ∈X P T ′ t =1 λα t k z t ( x ) k (cid:3) ,where the α t are simply coefficients to keep track of how many trades occurred between the buy andsell of noice trade t . We can proceed as follows: N T L ( λ, T ′ ) ≤ E sup x ,...,x T ′ ∈X T ′ X t =1 λα t k z t ( x t ) k = λ T ′ X t =1 α t E (cid:20) sup x ∈X k z t ( x ) k (cid:21) = λ T ′ X t =1 α t K , where K is simply the constant E [sup x ∈X k z t ( x ) k ] where the expectation is taken over the Gaus-sian process generating the noise. It is well-known that the expected maximum of a Gaussianprocess is bounded [14], and thus boundedness of K follows from the fact that Y is finite. Thus,continuing from eq. (3) we obtain N T L ( λ, T ′ ) ≤ T ′ log T ′ λK as before, with this new K . Finally,the proof of Theorem 3 now goes through, as it only treats the mechanism from Theorem 2 as ablack box.We close by noting the similarity between the kernel adaptive market mechanism and tradi-tional learning algorithms, as alluded to in the introduction. As observed by Abernethy, etal. [2], the market price update rule for classical prediction markets resembles Follow-the-Regularized-Leader (FTRL); specifically, the price update at time t is given by p t = ∇ C ( q t ) = argmax w ∈ ∆( Y ) h w, P s ≤ t dq s i − R ( w ) , where dq s is the trade at time s , and R = C ∗ is the convexconjugate of C .In our RKHS setting, we can see the same relationship. For concreteness, let C x ( q ) = λ C ( λq ) for all x ∈ X , and let R : ∆( Y ) → R be the conjugate of C . Suppose further that each agent t purchases a basis contract df t = f x t ,y t , where we take a classification kernel k ′ (( x, y ) , ( x ′ , y ′ )) = ( x, x ′ ) { y = y ′ } . Letting dq t ( x ) = df t ( x, · ) ∈ R Y , the market price at time t is given by, p tx = argmax w ∈ ∆( Y ) (cid:28) w, X s ≤ t dq s ( x ) (cid:29) − λ R ( w )= argmax w ∈ ∆( Y ) (cid:28) w, X s ≤ t k (( x s , y s ) , ( x, · )) (cid:29) − λ R ( w )= argmax w ∈ ∆( Y ) (cid:28) w, X s ≤ t k ( x s , x ) y s (cid:29) − λ R ( w ) , where y is an indicator vector. Thus, the market price update follows a natural kernel-weightedFTRL algorithm, where the learning rate λ is the price sensitivity of the market. Motivated by the problem of purchasing data, we gave the first bounded-budget prediction mar-ket mechanism that achieves privacy, incentive alignment, and precision (low impact of privacy-preserving noise the predictions). To achieve bounded budget, we first introduced and analyzed atransaction fee, achieving a slowly-growing O ((log T ) ) budget bound, thus eliminating the arbi-trage opportunities underlying previous impossibility results. Then, observing that this budget stillgrows in the number of participants T , we further extended these ideas to design an adaptively-growing market, which does achieve bounded budget along with privacy, incentive, and precisionguarantees.We see several exciting directions for future work. An extension of Theorem 4 where Y need notbe finite should be possible via a suitable generalization of Claim 2. Another important directionis to establish privacy for parameterized settings as introduced by Waggoner, et al. [15], where in-stead of kernels, market participants update the (finite-dimensional) parameters directly as in linearregression. Finally, we would like a deeper understanding of the learning–market connection in non-parametric kernel settings, which could lead to practical improvements for design and deployment. References [1] J. Abernethy and R. Frongillo. A characterization of scoring rules for linear properties.In
Proceedings of the 25th Conference on Learning Theory , pages 1–27, 2012. URL http://jmlr.csail.mit.edu/proceedings/papers/v23/abernethy12/abernethy12.pdf .[2] Jacob Abernethy, Yiling Chen, and Jennifer Wortman Vaughan. Efficient market making via convexoptimization, and a connection to online learning.
ACM Transactions on Economics and Computation , 1(2):12, 2013. URL http://dl.acm.org/citation.cfm?id=2465777 .[3] Jacob Abernethy, Sindhu Kutty, Sébastien Lahaie, and Rahul Sami. Information aggregation in expo-nential family markets. In
Proceedings of the fifteenth ACM conference on Economics and computation ,pages 395–412. ACM, 2014. URL http://dl.acm.org/citation.cfm?id=2602896 .[4] Jacob D. Abernethy and Rafael M. Frongillo. A collaborative mechanism for crowdsourcing predictionproblems. In
Advances in Neural Information Processing Systems 24 , pages 2600–2608, 2011.[5] Jacob D. Abernethy, Rafael M. Frongillo, Xiaolong Li, and Jennifer Wortman Vaughan. AGeneral Volume-parameterized Market Making Framework. In
Proceedings of the FifteenthACM Conference on Economics and Computation , EC ’14, pages 413–430, New York, NY,USA, 2014. ACM. ISBN 978-1-4503-2565-3. doi: 10.1145/2600057.2602900. URL http://doi.acm.org/10.1145/2600057.2602900 .[6] A. Banerjee, X. Guo, and H. Wang. On the optimality of conditional expectation as a Bregman predictor.
IEEE Transactions on Information Theory , 51(7):2664–2669, July 2005. ISSN 0018-9448. doi: 10.1109/TIT.2005.850145.[7] T-H Hubert Chan, Elaine Shi, and Dawn Song. Private and continual release of statistics.
ACM Transac-tions on Information and System Security (TISSEC) , 14(3):26, 2011.
8] Rachel Cummings, David M Pennock, and Jennifer Wortman Vaughan. The possibilities and limitationsof private prediction markets. In
Proceedings of the 17th ACM Conference on Economics and Computa-tion , EC ’16, pages 143–160. ACM, 2016.[9] Cynthia Dwork and Aaron Roth.
The algorithmic foundations of differential privacy . Foundations andTrends in Theoretical Computer Science, 2014.[10] Cynthia Dwork, Moni Naor, Toniann Pitassi, and Guy N Rothblum. Differential privacy under continualobservation. In
Proceedings of the forty-second ACM symposium on Theory of computing , pages 715–724.ACM, 2010.[11] R. Frongillo, N. Della Penna, and M. Reid. Interpreting prediction markets: a stochastic ap-proach. In
Advances in Neural Information Processing Systems 25 , pages 3275–3283, 2012. URL http://books.nips.cc/papers/files/nips25/NIPS2012_1510.pdf .[12] Rafael Frongillo and Mark D. Reid. Convergence Analysis of Prediction Markets via RandomizedSubspace Descent. In
Advances in Neural Information Processing Systems , pages 3016–3024, 2015. URL http://papers.nips.cc/paper/5727-convergence-analysis-of-prediction-markets-via-randomized-subspace-descent .[13] L.J. Savage. Elicitation of personal probabilities and expectations.
Journal of the American StatisticalAssociation , pages 783–801, 1971.[14] Michel Talagrand.
Upper and lower bounds for stochastic processes: modern methods and classicalproblems , volume 60. Springer Science & Business Media, 2014.[15] Bo Waggoner, Rafael Frongillo, and Jacob D Abernethy. A Market Framework for Eliciting Pri-vate Data. In
Advances in Neural Information Processing Systems 28 , pages 3492–3500, 2015. URL http://papers.nips.cc/paper/5995-a-market-framework-for-eliciting-private-data.pdf . Private (Unbounded-Loss) Markets
In this section, we review the private prediction market construction of Waggoner et al. [15]. Weinclude proofs for completeness and clarity, as we focus on classic, complete cost-function basedmarkets here whereas that paper focused on “kernel markets” which required additional formalism.Our adaptive market will rely on this market construction and results.
Approach and notation.
In a private market, the designer chooses an initial “true” market state q (for convenience, we will assume q = 0 ) and announces a “published” market state ˆ q = 0 .When participant t = 1 , . . . , T arrives and requests trade dq t , the market maker updates the truemarket state q t = q t − + dq t , but does not reveal the true state to anyone. Instead, the marketmaker announces the published market state ˆ q t , which is some randomized function of all tradesand published market states so far. We assume that k dq t k ≤ according to some norm k · k , that is,each participant can buy or sell at most one “total” share. Let k · k ∗ denote the dual norm to k · k . Differential privacy.
The market mechanism can be viewed as a randomized function M thattakes as input a list of trades ~dq = dq , . . . , dq T and outputs a list of published market states ˆ q , . . . , ˆ q T . We will call it ( ǫ, δ ) -differentially private if changing a single participant’s trade doesnot change the distribution on outputs much: if for all ~dq and ~dq ′ differing only in one entry, and forall (measurable) sets of possible outputs S , Pr h M (cid:16) ~dq (cid:17) ∈ S i ≤ e ǫ Pr h M (cid:16) ~dq ′ (cid:17) ∈ S i + δ. The mechanism is ǫ -differential private if it is ( ǫ, -d.p. It is reasonable to treat ǫ as a constantwhose size controls the privacy guarantee, such as ǫ = 0 . . Meanwhile, δ is normally preferred tobe vanishingly small or , as a mechanism can leak the private information of all individuals with δ probability and still be ( ǫ, δ ) -differentially private.To be careful, we note that the market’s “full output” also includes that it sends each participanttheir payoff. However, this payoff is a function only of the public noisy market states and of thatparticipant’s trade. The payoff is assumed to be sent privately and separately, unobservable byany other party. By the post-processing property of differential privacy, a trader’s ( ǫ, δ ) -privacyguarantee continues to hold regardless of how the published market states are combined with anyside information, even including the full list of all other participant’s trades. (This can be formalizedusing the notion of joint differential privacy , but for simplicity we will not do so.) Tool 1: generalized Laplace noise.
Imagine that the market could first collect all T trades simul-taneously, then sum them and publish some ˆ q T , a noisy version of the market state q T = P Tt =1 dq t .In this scenario, there is only one output ˆ q T instead of a whole list of outputs ˆ q , . . . . The standard,simplest solution to protecting privacy would be to take the true sum q T and add noise from ageneralization of the Laplace distribution. The real-valued Lap ( b ) random variable has probabilitydensity x b e −| x | /b . In R d , given a norm k · k , we define the generalized Lap d ( b ) distributionto have probability density proportional to e k x k /b . In this case, releasing ˆ q = q + Lap d (1 /ǫ ) is ǫ -differentially private: Given q, q ′ with k q − q ′ k ≤ , the ratio of probability densities at any ˆ q is e ǫ k q − ˆ q k /e ǫ k q ′ − ˆ q k ≤ e ǫ k q − q ′ k ≤ e ǫ . (When the norm is L , this corresponds to independent scalar Lap (1 /ǫ ) noise on each of the d coordinates.) Note that it also satisfies a good accuracy guarantee,as the amount of noise required does not scale with T ; so with enough participants, this mechanismbecomes a very accurate indication of the “average” trade while still preserving privacy. Tool 2: continual observation technique.
Unfortunately, the above solution is not sufficient be-cause our market must publish a market state at each time step. One naive approach is to apply theabove solution independently at each time step, i.e. produce each ˆ q t = q t + z t where z t containsindependent Laplace noise. The problem is that each step reveals more information about a trade,for instance, dq participates in T separate publications. To continue preserving privacy, each z t must have a much larger variance, which makes the published market states very inaccurate. One could also modify our approach to allow arbitrarily large trades, but this would also require addingproportionally large noise in order to continue to preserve privacy. dq • dq • dq • dq • dq • dq • dq • dq • dq • dq • dq • dq • dq • dq • dq • dq • dq Figure 1:
Picturing the continual observation technique for preserving privacy [7, 10]. Each dq t is a trade.The true market state at t is q t = P tj =1 dq j and the goal is to release a noisy version ˆ q t Each arrow originatesat t , points backwards to s ( t ) , and is labeled with independent Laplace noise vector z t . Now ˆ q t = q t + z t + z s ( t ) + z s ( s ( t )) + · · · . In other words, the noise added at t is a sum of noises obtained by following the arrowsall the way back to . There are two key properties: Each t has only log T arrows passing above it, and eachpath backwards takes only log T jumps. A second naive approach is to add noise to each dq t just once, producing ˆ dq t = dq t + z t . Then set ˆ q t = P ts =1 ˆ dq t . The benefit to this approach is that it can re-use the noisy z t variables across timesteps, rather than re-drawing new noise each time. The problem is that, while each z t is small inmagnitude, there are many of them; for example, the final ˆ q T contains T pieces of noise, which addup to a very inaccurate estimate of the true market state. This contrasts with the first naive approach,in which each publication only includes one piece of noise, but that piece of noise is very large.The idea of the “continual observation” technique, pioneered by Dwork et al. [10] and Chan et al.[7], is to strike a balance between these extremes by re-using noise a limited number of times whilealso keeping each piece of noise small. Roughly, each publication ˆ q t will include a logarithmic (in t ) number of pieces of noise, each of which is only “logarithmically large”. Definition 5.
We define the private market mechanism for observation Z ∈ Z and securities φ : Z → R d using a cost function C with parameter λ . At each time t participant t arrives and proposestrade dq t with k dq t k ≤ , unobservable to all others. At most T participants may arrive. Let q t = P tj =1 dq j . Let z t = 0 and for all t ≥ , let z t ∼ Lap d (2 ⌈ log T ⌉ /ǫ ) . At each time t , themechanism publishes market state ˆ q t := q t + z t + z s ( t ) + z s ( s ( t )) + · · · + z , where s ( t ) is defined by writing the integer t in binary, then flipping the rightmost “one” bit to zero.Participant t is charged C (ˆ q t + dq t ) − C (ˆ( q ) t ) , unobservable to all others. When outcome Z isobserved, she is paid dq t · φ ( Z ) , unobservable to all others.We note that s (0) = 0 and s ( t ) < t for all t > . A convenient notation is to let ˆ q s ( t ): t := t X j = s ( t )+1 dq j + z t . Then we can define the mechanism recursively as ˆ q t = q t + z t + z s ( t ) + z s ( s ( t )) + · · · + z = ˆ q s ( t ): t + ˆ q s ( t ) . Remark.
Notice that λ has no impact on the construction of the market, in particular does notaffect the amount of noise to add. Intuitively, this is because the market is defined entirely in “sharespace”, while price sensitivity relates shares to prices. We will not need to discuss λ until we discussaccuracy of the prices, which is irrelevant to the proof of privacy.12 heorem 5 (Privacy) . Assuming that all trades satisfy k dq t k ≤ (under the same norm as used forthe generalized Laplace distribution), the private mechanism is ǫ -differentially private in the trades dq , . . . , dq T with respect to the output ˆ q , . . . , ˆ q T .Proof. We emphasize that this proof does not contain new ideas beyond the original continual ob-servation technique, but merely adapts them to this setting.We will imagine that the market publishes every partial sum it uses, i.e. ˆ q s ( t ): t for all time steps t . The actual published values ˆ q t are functions of these outputs, so by the post-processing propertyof differential privacy [9], it suffices to show that publishing each of these partial sums would be ǫ -differentially private.The idea is to treat each publication of the form ˆ q s ( t ): t as a separate mechanism, which has (claim1) a guarantee of ( ǫ/ ⌈ log T ⌉ ) -differential privacy. We then show (claim 2) that any one trade dq t participates in at most ⌈ log T ⌉ of these mechanisms. These two claims imply the result because,by the composition property of differential privacy [9], each trade is therefore guaranteed ⌈ log T ⌉ · ( ǫ/ ⌈ log T ⌉ ) = ǫ -differential privacy.First, we claim that each publication ˆ q s ( t ): t preserves ǫ/ ⌈ log T ⌉ differential privacy of each trade dq t ′ that participates, i.e. with s ( t ) < t ′ ≤ t . This follows from the definition of ˆ q s ( t ): t because,since k dq t ′ k ≤ , an arbitrary change in dq t ′ changes norm of the partial sum of trades by at most ,and z t is a draw from the generalized Lap d (2 ⌈ log T ⌉ /ǫ ) distribution with respect to the same norm.Second, we claim that each trade dq t ′ participates in at most ⌈ log T ⌉ different partial sums ˆ q s ( t ): t .To show this, we only need to count the time steps t where s ( t ) < t ′ ≤ t , in other words, integers t ≥ t ′ where zeroing the rightmost “one” bit gives a number less than t ′ .Without loss of generality, the the binary expansion of t is b m b m − . . . b j . . . for some m, j andthen s ( t ) has expansion b m b m − . . . b j . . . . Hence the condition s ( t ) < t ′ ≤ t implies that thebinary expansion of t matches that of t ′ from bits m to j , then has a one at bit j − , and has zeroesat all lower-order bits. Since m is fixed for t ′ , this can only happen once for each j , or at most m total times; and m ≤ ⌈ log T ⌉ because t ′ ≤ T . Lemma 2 (Accuracy of share vector) . In the private mechanism with L norm, d securities, and T time steps, we have with probability − γ , max t k q t − ˆ q t k ≤ √ d log ⌈ T ⌉ ǫ ln (cid:18) T dγ (cid:19) . Proof.
For each t , each coordinate i of q t − ˆ q t is the sum of at most ⌈ log T ⌉ independent variablesdistributed Lap (2 ⌈ log T ⌉ /ǫ ) . We will choose β such that each coordinate’s absolute value exceeds β with probability γT d ; there are d coordinates per time step and T time steps, so a union boundgives the result.Choose β such that, if Y is the sum of k = ⌈ log T ⌉ independent Lap ( b ) variables with b =2 ⌈ log T ⌉ /ǫ ) variables, then Pr[ | Y | > β ] = γ ′ . A concentration bound for the sum of k independent Lap ( b ) variables, Corollary 12.3 of Dwork andRoth [9] gives β ≤ √ b ln 2 γ ′ . Now choose γ ′ = γT d . To recap, each | q t ( i ) − ˆ q t ( i ) | ≤ β except with probability γ ′ = γT d , hence bya union bound this holds for all t, i except with probability γ , hence k q t − ˆ q t k ≤ dβ except withprobability γ .As mentioned above, the previous results (Theorem 5 and Lemma 2) do not depend on λ at all,because they do not mention the prices. We now ask what a “reasonable” choice of λ can be so thatthe prices are interpretable as predictions, i.e. the prices are “accurate”. In the parameters of that Corollary, we choose ν = b p ln(2 /γ ′ ) as we will have ln(2 /γ ′ ) > k . heorem 6 (Accuracy of prices) . In the private mechanism, let p t = ∇ C ( q t ) and let ˆ p t = ∇ C (ˆ q t ) .Then to satisfy k p t − ˆ p t k ≤ α for all t , except with probability γ , it suffices for the price sensitivityto be λ ∗ = α ǫ √ d ⌈ log T ⌉ ln(2 T d/γ ) . Proof.
By definition of λ , we have k p t − ˆ p t k ≤ λ k q t − ˆ q t k ≤ λ √ d ⌈ log T ⌉ ǫ ln (cid:18) T dγ (cid:19) (5)for all t except with probability γ , by Lemma 2. We now just choose λ so that (5) ≤ α . B Slowly-Growing Budget
We prove the incentive and budget claims in separate lemmas.
Lemma 3 (Incentive to trade) . In the private market with transaction fee α , a participant at time t having belief p with k p − ˆ p t k ∞ ≥ α can make a strictly positive expected profit by participating.Proof. Ignoring the transaction fee, the expected profit from purchasing dq (recall k dq k ≤ ) isprofit = h dq, p i + C (ˆ q t ) − C (ˆ q t + dq ) . Because C is convex, C (ˆ q t ) − C (ˆ q t + dq ) ≥ h∇ C (ˆ q t + dq ) , − dq i . Soprofit ≥ h dq, p − ∇ C (ˆ q t + dq ) i = h dq, p − ˆ p t i − h dq, ∇ C (ˆ q t + dq ) − ˆ p t i . By Hölder’s inequality and the definition of price sensitivity, h dq, ∇ C (ˆ q t + dq ) − ˆ p t i ≤ k dq k ∞ k∇ C (ˆ q + dq ) − ˆ p k ≤ k dq k ∞ λ k dq k ≤ λ. So we have max profit ≥ max dq : k dq k ≤ h dq, p − ˆ p t i − λ = k p − ˆ p k ∞ − λ ≥ α − λ> α as λ < α by construction. Because there exists a trade with expected profit strictly above α , thetrader has an incentive to pay the α transaction fee and participate. Claim 3 (Claim 1) . For each t , exactly b ( t ) traders arrive between the purchase and the sale ofbundle z t ; furthermore, q t sell − q t buy is exactly equal to the sum of these participants’ trades.Proof. Note that if we write t in binary, it has a one in the bit position log b ( t ) , followed by zeros.By definition of the algorithm, z t is sold at the next time t ′ > t where the bit log b ( t ) is flipped tozero. So we have t ′ − t = b ( t ) , so b ( t ) traders have arrived.Now we want to show that q t sell − q t buy is the sum of their trades, i.e. that every noise trade bundle z t ′ held by the trader before buying z t is still held at the moment of selling z t , and no other noise tradebundles are held at that time.Consider all of the noise bundles that were already held at time t (after selling the appropriatebundles at that time, but before purchasing z t ). By definition of the algorithm, these were purchasedat times s with b ( s ) > b ( t ) , so by the above discussion, they are not sold until times t ′ where thebits in positions log b ( s ) are flipped to zero, which cannot happen until after bit log b ( t ) is flippedand z t is sold. Meanwhile, every bundle purchased after z t is sold by, at the latest, the same timethat z t is sold, as they correspond to lower-order bits; and any sold at the same time as z t are soldfirst because they were purchased later. 14 laim 4 (Claim 2) . If the noise trader purchases and later sells z t , then her net loss in expectationover z t (but for any trader behavior in response to z t ), is at most λb ( t ) K where K = E k z t k .Proof. Given the noise trader’s bundle drawn is z t , her loss is: C ( q t buy + z t ) − C ( q t buy ) + C ( q t sell ) − C ( q t sell + z t ) . The first pair of terms represents the payment made to purchase z t (moving the market state from q t buy ); the second pair represents the payment to sell z t (moving the state to q t sell ). Claim 3 impliesthat k q t buy − q t sell k ≤ b ( t ) , as each trader can buy or sell at most unit of shares. Therefore, the netloss on bundle z t is at most E z t max q,q ′ : k q − q ′ k ≤ b ( t ) C ( q + z t ) − C ( q ) + C ( q ′ ) − C ( q ′ + z t ) Now, we have C ( q + z t ) − C ( q ) = Z x =0 ∇ C ( q + xz t ) · z t dx,C ( q + r + z t ) − C ( q + r ) = Z x =0 ∇ C ( q + r + xz t ) · z t dx. So the difference is Z x =0 (cid:10) ∇ C ( q + xz t ) − ∇ C ( q + r + xz t ) , z t (cid:11) dx ≤ Z x =0 λ k r k k z t k dx = λ k r k k z t k by definition of price sensitivity λ . We also have k r k ≤ k r k ≤ b ( t ) . This bound holds foreach outcome of z t and any behavior of the participants, so we conclude the lemma statement, thatexpected loss is bounded by λb ( t ) E k z t k . C Constant Budget Bound
Lemma 4.
Let
A, D be constants at least with AD ≥ . Then for all T ≥ A (ln( AD )) , wehave T ≥ A (ln( T D )) .Proof. Let T ∗ = 9 A (ln( AD )) . First, we prove the inequality for T ∗ : T ∗ = 9 A (ln( AD )) = A (3 ln( AD )) = A (cid:0) ln( AD ( AD ) ) (cid:1) . Now for all AD ≥ , we have AD ≥ AD ) , so T ∗ ≥ A (cid:0) ln (cid:0) AD (ln( AD )) (cid:1)(cid:1) = A (ln( T ∗ D )) , as desired. Now we wish to extend this to all T ≥ T ∗ . Compare the derivative of the left side, dTdT = 1 , with that of the right side: ddT (cid:16) A (ln( T D )) (cid:17) = 2 A ln( T D ) T ≤ T D ) ≤ at T = T ∗ . Now if the inequality holds for all T ′ ∈ [ T ∗ , T ) , then it holds for T as the left side onlyincreases more quickly than the right. So by transfinite induction, it holds for all T ≥ T ∗ .15 roof of Theorem 3. Fix the parameters ǫ and d throughout. Let λ ∗ ( T, α, γ ) be the price sensitivityparameter as a function of these variables given in Theorem 6. ǫ -differential privacy of the market follows by the post-processing property of differential privacy [9]because each stage k is differentially private for the participants who arrive in that stage. All infor-mation released after stage k depends on these participants only through the noisy market state atthe end of stage k , which is ǫ -d.p.To show the incentive guarantee, note that the transaction fee is always fixed at α , so the incentiveproof of Theorem 2 goes through immediately.To show the accuracy guarantee, note the the prices up to T (1) arrivals satisfy an α/ guarantee;therefore the starting prices of the new market are within α/ of what they would be without addednoise. The prices up to T (2) additional arrivals are within α/ α/ of what they would have been(since they begin within α/ and are designed to stay within α/ of this shifted goal); and so on,telescoping to at most α . Similarly, the chance of failure of any of these guarantees, by a unionbound, is at most γ/ γ/ · · · ≤ γ .Now we must show bounded worst-case loss, and how to set T ( k ) . We will choose T (1) to be aconstant and each T ( k ) = 4 T ( k − .We will claim two things:1. In the final stage k where not all participants arrive, the market maker’s loss is at most α T ( k ) .2. In each stage k that is completed (all T ( k ) participants arrive), the market maker’s profitfrom that stage is at least α T ( k ) .These together prove bounded worst-case loss: If at least one stage is completed, the total profit isin fact positive: it is positive from all but the last stage, whose loss is at most α T ( k ) ≤ α T ( k − which is smaller than the profit made in stage k − . If no stages are completed, i.e. fewer than T (1) participants arrive, then expected worst-case loss is bounded by B ′ λ (1) . This gives a budget bound of B ′ λ (1) , which will be computed below. Proof of (1).
First, we must prove that the worst-case loss in stage is at most α T (1) . In doingso, we will explicitly compute a sufficient T (1) and this worst-case loss. Then, we must show thesame fact for all other stages.The worst-case loss in stage , by Theorem 2, is B = B ′ λ (1) = B ′ √ d ⌈ log T (1) ⌉ ln (cid:0) T (1) d/γ (cid:1) α ǫ ≤ B ′ √ d (cid:0) ln (cid:0) T (1) d/γ (cid:1)(cid:1) α ǫ . For convenience, set A ′ = B ′ √ dαǫ and set D = 4 d/γ . Then we have B ≤ A ′ (cid:0) ln( T (1) D ) (cid:1) ; toprove claim (1) for stage , we wish to pick T (1) such that B ≤ α T (1) . Setting A = α A ′ , we needto have A (cid:0) ln( T (1) D ) (cid:1) ≤ T (1) . By Lemma 4, this holds for T (1) = 9 A (ln( AD )) = B ′ √ d (cid:16) ln B ′ √ d γα ǫ (cid:17) α ǫ
16o for this choice of T (1) , we have claim (1) for stage . We note the budget bound is B ≤ α T (1) .Now we just show that T ( k ) increases faster than /λ ( k ) . T ( k ) = 4 T ( k − , but Bλ ( k ) = 4 √ B k d ⌈ log T ( k ) ⌉ ln (cid:0) T ( k ) d k /γ (cid:1) α ǫ = 2 4 √ B k − d ⌈ T ( k − ⌉ (cid:0) ln(8) + ln (cid:0) T ( k − d k − /γ (cid:1)(cid:1) α ǫ ≤ λ ( k − for sufficiently large T ( k − , i.e. if T (1) is a sufficiently large constant. So Bλ ( k ) grows more slowlythan T ( k ) and the inequality Bλ ( k ) ≤ α T ( k ) continues to hold. Proof of (2).
Let us lower-bound the profit in stage k if completed. By Inequality 4 (from Theorem2), that the market-maker profit if T ′ = T ( k ) participants arrive is T ( k ) (cid:16) c − Kλ ( k ) log T ( k ) (cid:17) − Bλ ( k ) = T ( k ) α − √ d ⌈ log T ( k ) ⌉ log T ( k ) ǫ ( α/ k ) ǫ √ d ⌈ log T ( k ) ⌉ ln(2 dT ( k ) k /γ ) ! − Bλ ( k ) = αT ( k ) (cid:18) − log T ( k ) k ) √ d ln(2 dT ( k ) k /γ ) (cid:19) − Bλ ( k ) . Recall that Bλ ( k ) ≤ α T ( k ) . We want to conclude that the profit in stage k is at least α T ( k ) , so wejust need to show that − log T ( k ) k ) √ d ln(2 dT ( k ) k /γ ) − ≥ . The fraction is decreasing in k , so it suffices to achieve this for k = 1 , d = 1 , and γ = 1 , where wehave log T (1) T (1) ) ≤ ..