Design and Analysis of a Synthetic Prediction Market using Dynamic Convex Sets
Nishanth Nakshatri, Arjun Menon, C. Lee Giles, Sarah Rajtmajer, Christopher Griffin
DDesign and Analysis of a Synthetic Prediction Market usingDynamic Convex Sets
Nishanth Nakshatri ∗ Arjun Menon ∗ C. Lee Giles † Sarah Rajtmajer † Christopher Griffin ‡ January 7, 2021
Abstract
We present a synthetic prediction market whose agent purchase logic is defined using a sigmoid trans-formation of a convex semi-algebraic set defined in feature space. Asset prices are determined by alogarithmic scoring market rule. Time varying asset prices affect the structure of the semi-algebraic setsleading to time-varying agent purchase rules. We show that under certain assumptions on the underlyinggeometry, the resulting synthetic prediction market can be used to arbitrarily closely approximate a bi-nary function defined on a set of input data. We also provide sufficient conditions for market convergenceand show that under certain instances markets can exhibit limit cycles in asset spot price. We providean evolutionary algorithm for training agent parameters to allow a market to model the distribution ofa given data set and illustrate the market approximation using two open source data sets. Results arecompared to standard machine learning methods.
Prediction markets in their current form trace their roots to the original studies by Hanson [1–4] and sincethen have been studied and used extensively [5–11]. For a survey of work in this area through 2007 see [12].In these markets, assets corresponding to future events (e.g., elections [13], sports outcomes [14] etc.) canbe bought and sold thereby manipulating underlying asset prices. These asset prices can be interpreted asprobabilities [7, 15] thereby providing a mechanism for event forecasting. Recent applications of predictionmarkets include forecasting infectious disease activity [16], evaluating scientific hypotheses [17], predictingthe reproducibility of scientific work [18], and aggregation of employee wisdom in a corporate setting [19,20].In practice, many of these markets have been remarkably successful in efficiently aggregating informationabout uncertain future events [21]. There are a number of compelling explanations for this. Financialstakes incentivize participants to search for better information [22] and the forecasts of more confidentagents are weighted more heavily, where confidence is measured as willingness to risk more money [23].The efficient markets hypothesis suggests that the market price reflects available information at least aswell as any competing method [24], although some have suggested that this hypothesis is not upheld inprediction markets [7]. Work has explored specific concerns about liquidity, price manipulation, outcomemanipulation, bias, and their respective impacts on market efficiency [15, 25–29]. A separate thread of thisresearch has studied the accuracy of prediction markets based on real versus play money, to disentanglethe specific role of financial incentives (see, e.g., [6, 30–32]). The arrival of blockchain technologies hasfacilitated the development of decentralized prediction markets (e.g., [33–35]), which benefit from the trustand transparency inherent in these ownerless peer-to-peer systems. Blockchain-based prediction marketsoffer anonymity for their traders [36,37], support broad participation, and reduce single points of failure [38].Design of decentralized prediction markets is an ongoing area of research [39–41]. ∗ Dept. of Computer Science and Engineering, Pennsylvania State University, University Park, PA 16802 † College of Information Sciences and Technology, Pennsylvania State University, University Park, PA 16802 ‡ Applied Research Laboratory, Pennsylvania State University, University Park, PA 16802 a r X i v : . [ c s . C E ] J a n ver the last decade, a body of work has emerged on so-called artificial (equivalently, synthetic) predictionmarkets. These are numerically simulated markets populated by artificial participants (agents) for thepurpose of supervised learning of probability estimators [42]. Like their human-populated counterparts,artificial prediction markets have found a number of applications, including lymph node detection from CTscans [43] and early stage detection of epidemics from crowd-sourced data [44]. The theoretical promise ofartificial markets was first explored by Chen and colleagues [45–47]. They highlight the deep mathematicalconnections between prediction markets and learning, demonstrating that any cost function based predictionmarket with bounded loss can be interpreted as a no-regret learning algorithm [46]. And, that every convexcost function based prediction market can be interpreted as a Follow the Regularized Leader algorithm witha convex regularizer [47].In an initial construction put forward by Barbu and Lay [42] patterned after the Iowa Electronic Markets[5], each agent is represented as a budget and a simple betting function. During training, each agent’sbudget is updated based on the accuracy of its prediction for each training data point. The contract pricefor an outcome is an estimator of its class-conditional probability. These markets, authors found, were ableto outperform random forest and implicit online learning in benchmark classification tasks. In follow-upwork [48], the same authors generalized the market framework to support regression and reported similargains in performance. Storkey and colleagues [49, 50] develop an artificial prediction market with a differentmarket mechanism, the so-called machine learning market. In their formulation, each agent purchasescontracts for possible outcomes in order to maximize its own utility function. The equilibrium price of thecontracts is computed by an optimization procedure. The market is shown to outperform standard classifierson a number of machine learning benchmarks. A 2014 extension of this work [51] models agents using staticrisk measures. The authors demonstrate that the resulting market approaches a global objective, formallyasserting the potential of the market to solve problems in machine learning. More recently, authors haveproposed continuous artificial prediction markets [52] for online regression. These markets consider agentswith adaptive trading strategies, using reinforcement learning to dynamically identify actions that maximizetheir own reward.In this paper we study synthetic prediction markets in which the agents’ purchase logic is governed bytime-varying semi-algebraic sets. For the purposes of this work, we focus on convex semi-algebraic setsdefined by ellipsoids in R n . Time variation of the set volume is governed by asset prices in the market.Agents specialize in the purchase of a single asset class and will only purchase an asset at time t if an inputfeature vector is contained in the (time-varying) set defining the agent. We show the following:1. Given an arbitrarily large but finite labeled data set, we show how to construct a market that willperfectly assign to each input the appropriate output. This allows us to derive a form of universalapproximation for our market structure.2. We provide a sufficient condition in terms of the underlying geometric structures for a market toconverge to a single final price for all assets.3. We show that the market can exhibit limit cycles and these limit cycles correspond to input data thatlie near decision boundaries of agents.4. We develop an evolutionary algorithm for training agent behavior in a market to represent a set ofinput data.5. We illustrate this algorithm using three open source data sets.Our results are complementary to the existing synthetic prediction market literature and establish a geometricfoundation for building more complex prediction markets.The remainder of this paper is organized as follows: In Section 2 we discuss the synthetic predictionmarket model and establish relevant notation. Theoretical results on the prediction market are establishedin Section 3. We discuss an algorithm for training a market to classify samples from a specific data setin Section 4. In Section 5 we show empirical results on three open source machine learning data sets.Conclusions and future directions of research are presented in Section 6.2 Binary Market Model
Let Z + be the positive integers. Assume we have a binary option market with the two options denoted asAssets 0 and 1. Assume q t = ( q t , q t ) ∈ Z units of (Asset 0, Asset 1) at time t have been sold. A (binaryoption) market [15] M consists of a set of agents A = { a , . . . , a n } who buy (and sell) Assets 0 and 1 usingpolicies { γ , . . . , γ n } . If agent purchase policy γ i is conditioned on exogenous information x ∈ D ⊆ R n then, γ i : ( q t , x ) (cid:55)→ ( r , r ) and Agent i purchases r units of Asset 0 and r units of Asset 1, thus causing a stateupdate. When the market is conditioned on x ∈ D we denote it M x .Assuming time passes discretely (is epochal) and we have an input x ∈ D , market M x is a dynamicalsystem ( Z , Z + , Γ x ) where the dynamic Γ x : Z → Z arises from the interaction of the individual policies { γ , . . . , γ n } and the conditional information x . At any time t , the state q t can be mapped into a pair ofasset prices p t = ( p t , p t ) that may be used in the policies of the agents in place of q t . For the remainder of this paper, we will assume that Γ x is fixed when given x and that an initial state q isgiven. We use the Logarithmic Market Scoring Rule (LMSR) [53] to aggregate estimates from a set of agents A = { a , . . . , a n } and determine asset prices. Given state ( q t , q t ), the current asset prices are computedusing LMSR: p t = exp ( βq t )exp ( βq t ) + exp ( βq t ) p t = exp ( βq t )exp ( βq t ) + exp ( βq t ) . This is the softmax function (Boltzmann distribution with constant β = k/T for fixed k and T ) of the inputs( q t , q t ). The β term is a liquidity factor [54] that adjusts the amount the price will increase or decreasegiven a change in the asset quantities. By using a Boltzmann distribution, the prices can be interpreted asprobabilities.The true asset purchase prices (trade costs) are not given by p t , since LMSR incorporates a marketmaker cost. The trade costs are given by: κ t (∆ q ) = 1 β log (cid:18) exp[ β ( q t + ∆ q )] + exp[ βq t ]exp[ βq t ] + exp[ βq t ] (cid:19) κ t (∆ q ) = 1 β log (cid:18) exp[ βq t ] + exp[ β ( q t + ∆ q )]exp[ βq t ] + exp[ βq t ] (cid:19) , where ∆ q i is the change in the quantify of Asset i as a result of purchases defined by Γ x .Let P ( x , t ) = p t ( x ) assuming fixed q and Γ x . The market converges to a price pair ¯ p if:lim t →∞ P ( x , t ) = ¯ p . (1)Convergence is not necessarily guaranteed in all markets, however for the markets we consider, we will showsufficient conditions for convergence to occur.Let φ : D ⊆ R n → [0 ,
1] be a binary function. Our objective is to construct Γ x , which defines a market M and agents A , so that: (cid:90) D | φ ( x ) − ¯ p ( x ) | d x < (cid:15), (2)where ¯ p ( x ) is the long-run price of Asset 1 and (cid:15) > L error when the price of Asset 1 is used as an approximation function for φ . We make this more precisein subsequent sections. 3 .2 Agent Purchase Policies Let f ( x ; θ ) be a quasi-concave function parameterized by θ with maximum at . By this we mean a functionthat satisfies the inequality: f ( λ x + (1 − λ ) x ; θ ) ≥ max { f ( x ; θ ) , f ( x ; θ ) } . (3)If Θ is a positive definite, diagonal matrix, then the quadratic function: g ( x ; θ ) = 1 − x T Θx = 1 − (cid:88) j θ i x i (4)is such a function and the set: E Q = { x ∈ R n : g ( x ; Q ) ≤ } (5)is an ellipsoid centered at and oriented along the standard basis.For the chosen quasi-concave function, define the translated function: f ( x ; h , θ ) = f ( x − h ; θ ) (6)In terms of the quadratic function this is just: g ( x ; h , Θ ) = ( x − h ) T Θ ( x − h ) . (7)Under these assumptions, Θ defines a simple local metric that is used to determine how close the conditioningpoint x is to a reference point h .Assume we are given a set of labeled training data H = { h i } Ni =1 with labels y = { y i } Ni =1 with y i ∈ { , } .For each data point h i in H (or possibly an appropriate subset of H ) with label y i define Agent i who buys only Asset y i ( i = 0 , i specializes in buying y i . Given an input featurevector x , Agent i estimates the value of Asset y i using the formula: π it ( x , p y i t ; h i ; θ i , α i , w ip ) = σ [ α i · f ( x ; h i , θ i ) + θ i + w ip ( p y i t − p y i )] , (8)where p y i t is the price of Asset y i at time t , θ is a bias, α is a scaling factor and σ is the logistic sigmoidfunction . When using an ellipsoidal function, the exact formula is: π it ( x , p y i t ; h i , θ i , α i , w ip ) = σ α i · − (cid:88) j θ ij ( x − h ij ) + w ip ( p y i t − p y i ) + θ i . (9)We note that if θ ij = 0, then the ellipsoid structure is replaced (effectively) with a cylinder in R n .We assume Agent i can only buy one unit of Asset y i at a time (per epoch). The agent logic defining γ i is then:1. For ∆ q y i = 1, if 1 κ y i t (cid:0) π it (cid:0) x , p y i t ; h i ; θ i , α i , w ip (cid:1) − κ y i t [∆ q y i ] (cid:1) ≥ τ, then the agent purchases a single unit of Asset y i . Here τ ∈ [0 ,
1) determines the opportunity costconsidered by the agent. When τ = 0, the agent purchases an asset precisely when it has sufficientfunds and when it’s estimated price is higher than the actual asset price.2. Otherwise, the agent purchases nothing.For our model, each agent only buys when the conditioning data x ∈ D is close enough (in the derivedmetric) to its initialized data point h i . Thus, we are using the data set H to construct a covering of the set D and then using that covering to construct the market and its dynamics. A unit step function could be substituted with minimal change to the sequel. Properties of the Market
In this section, we study the theoretical properties of markets in which agents have unlimited funds.
Proposition 1.
Let H = (cid:8) h i (cid:9) Ni =1 be a finite but arbitrarily large data set with labels y = { y i } Ni =1 . Assumethe data are separable; i.e., if h i = h j , then y i = y j . For all (cid:15) > , there is a market M with agents A = { a , . . . , a N } such that for all i = 1 , . . . , N : lim t →∞ | p ( h i , t ) − y i | < (cid:15), (10) where p ( x , t ) is the price of Asset 1 in the market (the market spot price).Proof. Set τ = 0. The fact that H is finite implies there is a set of open spheres centered at h , . . . , h N withradii r , . . . , r N so that: h j ∈ B r i ( h i ) ⇐⇒ h j = h i (11)From Eq. (9), for all i and j , define θ ij = 1 /r i . For all i set w ip = θ i = 0. Assume that Agent i purchasesonly Asset y i . For Agent i using Eq. (9) the estimated price given h i is constant and given by: π i ( h i ) = π it ( h i , p y i t ; h i , θ i , α i , w ip ) = σ ( α i ) > . (12)Likewise, it is clear that for h j (cid:54) = h i : π i ( h j ) = π it ( h j , p y i t ; h i , θ i , α i , w ip ) < , (13)since by construction: 1 − (cid:88) k (cid:16) h jk − h ik (cid:17) r i < . Set α i = α so that (by choice of α ) for all i, j : π i ( h i ) > − δπ i ( h j ) < δ, for a δ ∈ (0 , (cid:15)/ α must exist because σ is monotonic and bounded between 0 and 1. When h i isused as the market input (i.e., x = h i ), then Agent i will purchase one share of Asset y i per epoch until thefirst time t (1) when: 1 − δ < π i ( h i ) < κ y i t (1) . Choose β small enough to ensure that at this point:1 − δ < p y i t (1) = e βt (1) e βt (1) < κ y i t (1) . (14)There are two possibilities. Case I : For all j : π j ( h i ) < δ < κ − y i t (1) . In this case, the market converges to price p y i t (1) > − δ > − (cid:15) as required. Case II:
There is at least one j so that π j ( h i ) > κ − y i t (1) . t (1) all such agents will purchase shares of asset (1 − y i ) and will continue to do so until t (2) at whichpoint either Case I holds or Agent i purchases again. In each case, assume β is chosen small enough so thatat time t (2) : p y i t (2) > − δ. (15)This ensures that the purchases of the other agents cannot drive the price too far from 1 − δ . Such a β mustexist because asset price moves are monotonically decreasing in β . Since π i ( h i ) and π j ( h i ) are fixed for alltime and H is finite, a smallest fixed value of β must exist to make Eqs. (14) and (15) true for all time. (SeeFig. 1.) We repeat the above logic to see that for time t ≥ t (1) , p y i t ∈ (1 − δ, − δ ) and Eq. (10) holds. Thiscompletes the proof.Figure 1: We illustrate the difference between the spot price p t and the purchase price κ t (∆ q ) for asset oneunder varying values of q and q with ∆ q = 1. The value of β = 1 / β decreases, the difference κ t (∆ q ) − p t →
0. Thus ensuring Eqs. (14) and (15) .Using the prior result, it is straightforward to see that if D ⊂ R n is a simply connected closed and boundedset and χ D ( x ) is its characteristic function, then if (cid:15) >
0, there is a market M with agents A = { a , . . . , a N } (for some possibly large N) so that: (cid:90) D (cid:12)(cid:12) χ D ( x ) − ¯ p ( x ) (cid:12)(cid:12) d x < (cid:15). (16)To see this, choose a large but finite sample of points from D and add to this an appropriately large sampleof points near the boundary of D . Call this set H and apply an argument like the one for Proposition 1 toconstruct the market. From this we conclude: Proposition 2. If D is a finite union of simply connected closed and bounded subsets of R n and (cid:15) > , thenthere is a market M and a finite (but large) set of agents so that Eq. (16) holds. We effectively illustrate Proposition 2 in Section 5.3.
Let: Ω it = (cid:26) x ∈ R n : 1 κ y i t (cid:0) π it (cid:0) x , p y i t ; h i ; θ i , α i , w ip (cid:1) − κ y i t [∆ q y i ] (cid:1) ≥ τ (cid:27) (17)the following proposition provides a sufficient condition for the convergence of the market price to a singlevalue. 6 roposition 3. Consider a market M with agent set A = { a , . . . , a N } and a fixed β , τ . Given an input x ∈ R n , if there is a time t ∗ and an index set I ∗ = { i , . . . , i k } ⊂ { , . . . , N } so that for all t ≥ t ∗ : x ∈ (cid:92) i ∈ I ∗ Ω it , (18) and if j (cid:54)∈ I ∗ , then x (cid:54)∈ Ω jt , then the price p t converges to a fixed value.Proof. Suppose there is a t ∗ and I ∗ = ∅ . Then no agent purchases occur at time t ≥ t ∗ and the market price p t remains constant at the value p t − . If I ∗ is not empty, then assume there are r ≥ I ∗ and s ≥ I ∗ . Then for all time the spot price for Asset1 is given by: p t = exp (cid:2) β (cid:0) q t − + rt (cid:1)(cid:3) exp (cid:2) β (cid:0) q t − + rt (cid:1)(cid:3) + exp (cid:2) β (cid:0) q t − + st (cid:1)(cid:3) , (19)because at all future times the agents in I ∗ will purchase 1 unit of the appropriate asset. Taking the limitat t → ∞ yields: ¯ p = s > r s < r exp ( βq ) exp( βq )+exp( βq ) if r = s (20)This completes the proof.We note that when each agent is given a finite bank account, then convergence of the market is ensuredand the decision logic must be amended to include a test for sufficient funds.It is easy to construct an example in which the market does not converge to a fixed point. To see this,consider a market with a two dimensional feature space and two agents with h = (0 ,
0) and h = (2 , x = (1 . , τ = 0, β = 1 / i = 1 , α i = 3, w ip = 2. If we assume both agents have r ij = 1 .
015 (i.e., agent geometry is circular), then this market will oscillate in price forever as illustrated inFig. 2. The oscillation in the price is caused by the oscillation in the geometric structure of the sets Ω t and A ss e t P r i c e Figure 2: An example of an oscillating market price for an input near the decision boundary.Ω t . As the market price varies in time, each agent oscillates between determining the price is too high orlow enough to purchase. Thus, when input information is close to a decision boundary we see that marketprices may exhibit a limit cycle. Establishing sufficient conditions for the emergence of a limit cycle in themarket is left to future work. However, as we have illustrated limit cycles will emerge when input (test)points are near multiple agent decision boundaries in feature space and thus can indicate indecision if themarket is used as a machine learning model. 7 Training Agents within a Market
In this section we discuss a practical implementation of the prediction market described above and detail amethod to train such a market to approximate a data set. For practical purposes, we make three simplifyingimplementation changes:1. We assume time is finite. That is, the market will terminate after a fixed large time.2. We assume all agents have a finite bank account.3. We assume that agents recurrently arrive at the market to buy assets with inter-arrival times governedby an exponential distribution. Thus not all agents interact with the market simultaneously.The third assumptions is made to increase the execution speed of the market and to ensure a sufficientnumber of training epochs can be executed in a reasonable amount of wall-clock time.
Let each training data-point be denoted as ( x i , y i ), where y i denotes the output label. Let m be the totalnumber of training data-points. Define: C k = (cid:8) i ∈ { , . . . , m } : y i = k (cid:9) , (21)for k = 0 ,
1. Training will proceed in batches. Assume a batch size of b , where b < m , to denote the numberof data-points used to train the model in one pass. Thus, there will be a total of (cid:100) m/b (cid:101) batches. A set of n Agents are initialized for every data-point x i in a batch B j where i ∈ [1 , b ] and j ∈ [1 , (cid:100) m/b (cid:101) ]]. The agentsare initialized as hyperspheres centered at h i = x i . To determine the initial radius, let r i = arg min j ∈C yi (cid:13)(cid:13) x i − x j (cid:13)(cid:13) r i = 12 · arg min j ∈C − yi (cid:13)(cid:13) x i − x j (cid:13)(cid:13) . These are the distances to the nearest point with similar classification and half the distance to the nearestpoint with opposite classification. Then set: p i = max (cid:8) r , min (cid:8) r i , r i (cid:9)(cid:9) q i = max (cid:8) r (cid:48) , r i (cid:9) where r and r (cid:48) are default values. The radius of the hyper-sphere is initialized with: r i ∼ U ( p i , q i ) , where U is a uniform distribution. That is, we model each agent with an ellipsoid so all axial radii areinitialized to r i . The initial value for w ip is chosen from a standard normal distribution for each i . Finally,if Agent i is centered at x i = h i with class y i , then that agent will only purchase assets of Class y i . Each market run is parameterized by an input feature vector x shared by all agents. This feature vectoris used in agent purchase logic. Agents are initialized with a finite bank. During an execution of themarket, each agent is seeded with an initial time it will interact with the market drawn from an exponentialdistribution. The next time of execution is set when the agent interacts with the market and uses the sameexponential distribution. All agents have a common exponential distribution. Agents buy assets accordingto the decision logic discussed above and keep track of purchased assets and the price paid. There is a globalclock that is updated to determine when agents participate. At market completion (after a fixed time haspassed), agent profits and losses are calculated assuming assets that match the ground truth class y i areworth 1 and other assets are valued at 0. 8 .3 Evolutionary Algorithm The evolutionary algorithm defined below is used to identify parameters θ i , w ip , and α . For the purposesof this work, we assume that θ i = 0 is fixed for all i , we set β = 1 /
100 and τ = . Optimizing theseparameters is a subject of future work. Evolutionary AlgorithmInput:
Feature vectors X = (cid:8) x i (cid:9) Ni =1 , ground truth labels (cid:8) y i (cid:9) Ni =1
1. For each data point x i with label y i create n agents centered at h i = x i and an initial random radius r i ∼ U ( p i , q i ), a random scale parameter α i ∼ U (0 . ,
5) and a random w ip ∼ N (0 , y i .2. Run N markets one for each input x i ∈ X .3. For each market, sort all agents into three groups (i) those that did not participate, (ii) those thatmade a profit and (iii) those that had a loss.(a) For each center h i :i. If no agent with center h i participated, continue.ii. Among the agents who participated retain l < n agents who had the highest profit (or lowestloss).iii. Delete the n − l under-performing agents.iv. Create n − l new agents from the agent pool centered at h i using mutation and crossover ofthe parameters α , θ and w p . Specifically, mutation is carried out as follows:A. Compute σ = 2 (cid:114) n Σ mi =1 (cid:0) y i − ¯ p y i (cid:1) . (22)B. Update r i ← r i + σ · U ( p i − r i , q i − r i ).C. Update w ip ← w ip + σ · N (0 , g generations.Because each agent is modeled by an ellipsoid with a finite volume, not every agent will participate in everymarket. In particular, if x i (cid:54)∈ Ω it for any time t , then Agent i will not participate. Of those agents that doparticipate in a given market, those that are most successful are preserved and replicate with mutation andcrossover. The mutation rate is controlled by the current root mean-square error of the approximation. Asthis value decreases, the mutation decreases. This section discusses the results obtained by the application of the proposed market model on standarddatasets such as IRIS Dataset and Heart Disease Dataset. We also apply the model to perform the recordlinkage task of disambiguating inventor records from the USPTO PatentsView database as a real-worldapplication usecase. For all the experiments, we have chosen n = 5 (agent replicants), l = 3 (retainedagents) and g = 20 (generations). We study the standard IRIS data set [55], which consists of features describing three species of iris plants -Iris setosa, Iris virginica and Iris versicolor. The data set contains 50 instances of feature vectors from eachclass. It is known that Iris Setosa is linearly separable from the other two classes. However, Iris Versicolorand Iris Virginica are not linearly separable from each other. We use four attributes, length and width ofsepals and petals, to classify an instance into one of the three classes.The proposed market model is generalized to be a binary classifier. However, the dataset consists ofthree classes. Therefore, we take the union of two classes and train the model on the one-against-two binaryclassification problem. We used a train-test split of 75:25.9 nion of Iris Setosa and Iris Versicolor
We combined the two classes, Iris Setosa and Iris Versicolor,and represented them as Class 0. Class 1 was composed of data from to Iris Virginica. A test accuracy of94.6% was obtained in this case. A detailed analysis is shown in Table 1.
Class Precision Recall F1-Score
Class 0 (Setosa/Versicolor) 1.00 0.91 0.95Class 1 (Virginica) 0.88 1.00 0.95Table 1: Shows the test performance when instances of Iris Setosa and Iris Versicolor are combined togetheras Class0.
Union of Iris Setosa and Iris Virginica
We combined the instances of Iris Setosa and Iris Virginicaas Class 0. Class 1 contains data from Iris Versicolor. A test accuracy of 97.29% was observed and Table 2shows a detailed analysis.
Class Precision Recall F1-Score
Class 0 (Setosa/Virginica) 0.96 1.00 0.98Class 1 (Versicolor) 1.00 0.93 0.96Table 2: Shows the test performance when instances of Iris Setosa and Iris Virginca are combined togetheras Class0.
Union of Iris Versicolor and Iris Virginica
We combined the instances of Iris Versicolor and IrisVirginica as Class 1. Class 0 consists of data from Iris Setosa. A test accuracy of 100.0% was observedTable 3 shows a detailed analysis. We have to note that instances of Iris Setosa are linearly separable fromthe other two classes and thus, the model is able to separate the two classes with 100% accuracy in this case.
Class Precision Recall F1-Score
Class 0 (Setosa) 1.00 1.00 1.00Class 1 (Versicolor/Virginica) 1.00 1.00 1.00Table 3: Shows the test performance when instances of Iris Setosa and Iris Virginca are combined togetheras Class0.
This is a publicly available dataset [56] provided by UCI. There are four databases available for use withinthe dataset. Published experiments in Machine Learning use the Cleveland database with a maximum of14 of the 76 available attributes which are known to be considerably linked to heart disease. We use thefollowing 14 numerical attributes to train the market model to classify patients to one of the targets; presenceof heart disease, no heart disease.1. Age2. Sex: male, female3. Chest pain type: typical angina (angina), atypical angina (abnang), non-anginal pain (notang), asymp-tomatic (asymp)4. Trestbps: resting blood pressure on admission5. Chol: serum cholestrol 10. Fbs: indicates whether fasting blood sugar is greater than 120 mg/dl7. Restecg: normal(norm), abnormal(abn): ST-T wave abnormality, ventricular hypertrophy (hyp)8. Thalach: maximum heart rate achieved9. Exang: exercise induced angina10. Oldpeak: ST depression induced by exercise relative to rest11. Slope: upsloping, flat, downsloping: the slope characteristics of the peak exercise ST segment12. Ca: number of fluoroscopy colored major vessels13. Thal: normal, fixed defect, reversible defect - the heart status14. Class/target labelThe data set has a total of 303 data points. To evaluate the performance of the market (M), we split thedata using an 80%-20% ratio. The market was tested on 20% of the randomly sampled data. A total of60 data points were used for testing the model. For one of the randomly chosen split, we obtained a testaccuracy of 86.66%. The confusion matrix associated with the test data is shown in Fig. 3. + H D U W ' L V H D V H 1 R + H D U W ' L V H D V H 3 U H G L F W H G + H D U W ' L V H D V H 1 R + H D U W ' L V H D V H $ F W X D O Figure 3: Confusion Matrix obtained for the test set.The obtained results from the market model are compared with the output obtained from a RandomForest (RF) classifier for the same split. The RF classifier obtained a test accuracy of 96.66%. Table 4compares the F1-Score obtained for both the models. We see that Random Forest outperformed the marketin this case.
Model No Heart Disease (%) Heart Disease (%)RF
96 97 M
84 89Table 4: Shows the F1 scores for each class for both the classifiers.To measure the sensitivity of the model with respect to the variation in inputs, we performed the exper-iment with six randomly sampled data splits. A train-test ratio of 80%-20% was retained for all the splits.Fig. 4 shows the comparison of the Market model with RF Classifier. We observe variations in the twomodels with respect to changing input data. The RF classifier outperformed the market in five out of sixcases. Market performance was comparable to RF classifier for the fifth split. Fig. 5 shows the box plot ofF1-scores associated with each class for the two models.The lower performance of the market can be attributed to a lack of generalization using the underlyinggeometry. The use of simple geometric agents allows us to quantify this. Fig. 6 shows the number of markets11 ' D W D ) R O G $ F F X U D F \ &