Does an artificial intelligence perform market manipulation with its own discretion? -- A genetic algorithm learns in an artificial market simulation
DDoes an artificial intelligence perform marketmanipulation with its own discretion?– A genetic algorithm learns in an artificialmarket simulation –
Takanobu Mizuta ∗ SPARX Asset Management Co. Ltd., Tokyo, Japan
Abstract
Who should be charged with responsibility for an ar-tificial intelligence performing market manipulationhave been discussed. In this study, I constructed anartificial intelligence using a genetic algorithm thatlearns in an artificial market simulation, and investi-gated whether the artificial intelligence discovers mar-ket manipulation through learning with an artificialmarket simulation despite a builder of artificial intel-ligence has no intention of market manipulation. Asa result, the artificial intelligence discovered marketmanipulation as an optimal investment strategy. Thisresult suggests necessity of regulation, such as obligat-ing builders of artificial intelligence to prevent artificialintelligence from performing market manipulation.
Who should be charged with responsibility for an arti-ficial intelligence (AI) having an accident and/or per-forming an illegal action have been discussed. In finan-cial sector, who should be charged with responsibilityfor an AI performing market manipulation have beendiscussed. Market manipulation is that some tradersartificially increase or decrease market prices to gaintheir profits, and is prohibited in many countries asunfair trades.Scopino indicated that when a human has built anAI trader without intention to perform market manip-ulation and the AI trader has actually performed mar-ket manipulation with its own discretion, the humanmay not be charged with responsibility in the presentregulation of the united states [1]. This means thateven though market prices are manipulated no one ischarged with responsibility. This is a big problem to ∗ [email protected], https://mizutatakanobu.com prevent keeping quality of markets.An AI trader must automatically learn impacts of itstrades to market prices in order to discover that mar-ket manipulation earns profit because own trades mustincrease or decrease market prices to perform marketmanipulation. An AI trader is usually evaluated bybacktesting, in which the profit is estimated if the AItrader were trading at some time using historical realdata of market prices. An AI trader cannot learn im-pacts of its trades to market prices because marketprices are fixed as real historical data in the backtest-ing. Therefore, an AI trader will not discover thatmarket manipulation earns profit when the AI traderuse backtesting as learning process. Then, we do nothave to worry that an AI trader performs market ma-nipulation with its own discretion without the human’sintention as long as using backtesting.In contrast, an artificial market simulation using akind of agent-based model [2] allows an AI trader tobe able to automatically learn impacts of its trades tomarket prices because in the simulation market pricesare changed by trades of an AI trader.In this study, as Fig. 1 shown, I constructed an AItrader using a genetic algorithm that learns in anartificial market simulation, and investigated whetherthe AI trader discovers market manipulation throughlearning despite a builder of the AI trader has no in-tention of market manipulation. A genetic algorithm is a calculation method approximatelysearching an optimal solution inspired by the evolution of lifeby the force of natural selection. Input values are represent asgenes, and surviving a gene that has higher adaptability (outputvalue) leads to obtain an optimal solution, that is the input valuethat emerges the highest output value. Goldberg wrote the greattext book [3] a r X i v : . [ q -f i n . T R ] M a y rder Exchange
Artificial MarketSimulation
Actions
By a GeneGene GeneticAlgorithm All Actions
Profit=Evaluation
AI agent(AIA)Gene
Artificial MarketAll ActionsEvaluation
GeneGene ・・・・・・・・ ・・・・・・・・
B S S ・・・・・ BB S S ・・・・・S S S S ・・・・・ BB B B ・・・・・ S N t Actions (Buy(B), Sell(S), No action(null)) N g Genes Inherit to new Generation after Crossover and MutationRepeat N e generations ・・・・・・・・ NormalAgent (NA)
Artificial Market
Artificial MarketAll ActionsEvaluationAll ActionsEvaluation
Fig. 1: My model
A human building an AI trader (builder) gives theAI trader candidates of trading strategies, and makesthe AI trader to learn which strategies and parametersearn more. This study focuses whether an AI tradercan discover market manipulation through learning de-spite the builder has no intention of market manipula-tion .Fig. 1 schematically shows a model of this study. AnAI trader that the builder intents no trading strategyis modeled using a genetic algorithm in which a geneincludes all trades. Each gene is evaluated in the artifi-cial market simulation. The artificial market includesan AI agent (AIA) that trades exactly same as onegene indicating. The gene is evaluated by AIA’s profit In reality, the builder always intents some kinds of strate-gies in the process of picking up and modeling candidates ofstrategies. In contrast, it is very important for this study thatthe builder has no intention of any strategies including marketmanipulation. Therefore, I do not intentionally modeled trad-ing strategies and my model directly searches for all the besttrades in an artificial market environment. Due to no modelsof trading strategies my model can not make any outputs in anout-sample, then no one can test my model in an out-sample. Iargue, however, that this study needs no evaluations in an out-sample because this study focuses whether an AI trader can dis-cover market manipulation through learning despite the builderhas no intention of market manipulation. This study does notaim to use my model in actual financial markets that are in anout-sample environment. in the artificial market simulation. The genetic algo-rithm search the gene most earns profit. This searchingcorresponds with what the AI trader learns how tradesearns profit.Of course, trades of the AIA impact market pricesin the artificial market, but for the purpose of compar-ison, I also investigated the case without the impactsto market prices (backtesting).In the following, at first I explain the artificial mar-ket simulation evaluating each gene and then, I explainthe genetic algorithm searching the gene most earnsprofit.
In this study, I built an artificial market model addedan AIA to the artificial market model of Mizuta [2]In the model here, there is one stock. The stockexchange adopts a continuous double auction to de-termine the market price. In this auction mechanism,multiple buyers and sellers compete to buy and sellfinancial assets in the market, and transactions canoccur at any time whenever an offer to buy and an of-fer to sell match. The minimum unit of price change is δP . The buy-order price is rounded off to the nearestfraction, and the sell-order price is rounded up to thenearest fraction.The model includes n normal agents (NAs) and anAIA. Agents can short sell freely. The quantity of hold-2ng positions is not limited, so agents can take anyshares for both long and short positions to infinity.Agents always places an order for only one share. Iemployed “tick time” t that increase by one when anagent orders. To replicate the nature of price formation in actualfinancial markets, I introduced the NA to model a verygeneral investor. The number of NAs is n . First, attime t = 1, NA No. 1 places an order to buy or sellits risk asset; then, at t = 2 , , , , n , NAs No. 2 , , , , n respectively place buy or sell orders. At t = n + 1, themodel returns to the first NA and repeats this cycle.An NA determines an order price and buys or sells asfollows. It uses a combination of a fundamental valueand technical rules to form an expectation on a riskasset’s return. The expected return of agent j for eachrisk asset is r te,j = ( w ,j log P f P t − + w ,j log P t − P t − τ j − + w ,j (cid:15) tj ) / Σ i w i,j (1)where w i,j is the weight of term i for agent j and isindependently determined by random variables uni-formly distributed on the interval (0 , w i,max ) at thestart of the simulation for each agent. log is naturallogarithm. P f is a fundamental value and is a con-stant. P t is a market price that is the mid price (theaverage price of the highest buy order price and thelowest sell order price), and (cid:15) tj is determined by ran-dom variables from a normal distribution with average0 and variance σ (cid:15) . Finally, τ j is independently deter-mined by random variables uniformly distributed onthe interval (1 , τ max ) at the start of the simulation foreach agent .The first term of Eq. (1) represents a fundamentalstrategy: the NA expects a positive return when themarket price is lower than the fundamental value, andvice versa. The second term of Eq. (1) represents atechnical strategy using a historical return: the NAexpects a positive return when the historical marketreturn is positive, and vice versa. The third term ofEq. (1) represents noise.After the expected return has been determined, theexpected price is P te,j = P t exp ( r te,j ) . (2)An order price P to,j is determined by random vari-ables uniformly distributed on the interval ( P te,j − P d , P te,j + P d ) where P d is a constant. Whether tobuy or sell is determined by the magnitude relation-ship between P te,j and P to,j : When t < τ j , however, the second term of Eq. (1) is zero. when P te,j > P to,j , the NA places an order to buy oneshare, butwhen P te,j < P to,j , the NA places an order to sell oneshare . The remaining order is canceled after t c fromthe order time. Every δt tick time the AIA takes one of three actionsthat are buy one share (at the lowest sell order priceon the order book), sell one share (at the highest buyorder price on the order book) and no action . TheAIA takes actions N t = ( t e − t c ) /δt times throughthe whole one artificial market simulation, where onesimulation runs until tick time t e . The actions aregiven by one gene in the genetic algorithm as followingI will mention. Fig. 1 schematically shows a model of this study. AnAI trader that the builder intents no trading strategyis modeled using a genetic algorithm. The number ofgenes is N g . One gene has information of actions andthe number of actions that one gene has is N t . Eachaction is one of three actions that are buy one share,sell one share and no action. Each gene is evaluatedby profit of the AIA in an artificial market, in wherethe AIA trades every δt tick time same as N t actionsone gene indicating. When the AIA holds stocks atthe end of a simulation, the stocks are evaluated as P f . All artificial markets has exactly same NAs usingsame random numbers. Therefore, if the AIA tradessame, the artificial markets output same market pricesand same NAs’ trades. The top N ge genes that earned most are not changedand inherited to the next generation.Non top N ge genes are, with a probability of R c , re-placed to the crossed-over gene with two genes g and g that are randomly selected from the top N ge genes.In the crossover, first, all actions are replace with thoseof the gene g , and then from i th to i th actions ( i and i are randomly determined) are replaced withthose of the gene g . After crossovers, each action of When t < t c , however, to generate enough waiting orders,the agent places an order to buy one share when P f > P to,j , orto sell one share when P f < P to,j . But, the AIA dose not take any action before tick time t c to stabilize the simulations. As I mentioned at *4, the periodbefore t c is aimed to generate enough waiting orders. P r i c e Tick Timewith AIA without AIA
Fig. 2: Time evolution of market prices (mid prices)in the case with the AI agent (AIA) and without theAIAall the non top N ge genes is mutated with a probabil-ity of R m . The mutated action is changed with sameprobability to buy, sell or no action.This inheritance to the next generation is repeated N e times.At the first generation, all actions of all genes aredetermined with same probability to buy, sell or noaction. In this study, I set parameters for the artificial marketwith n = 900 , w ,max = 1 , w ,max = 100 , w ,max =1 , τ max = 1000 , σ (cid:15) = 0 . , P d = 1000 , t c = 2000 , δP =0 . , P f = 10000 , δt = 10. I ran simulations to t = t e = 10000. I set parameters for the genetic algorithmwith N t = ( t e − t c ) /δt = 800 , N g = 10000 , N ge =400 , R c = 0 . , R m = 0 . , N e = 1500. These lead N g × N e = 1 . × , this means that I have executed15 million simulation runs of the artificial market. Inthe following result, I used the AIA of the best geneat the final generation. Fig. 2 shows the time evolution of market prices (midprices) in the case with the AIA and without the AIA.The AIA amplified variation of market prices.Fig. 3 shows the time evolution of market priceswith the AIA and trading volume (positive and neg-ative number show buy and sell, respectively) aggre-gated within each 200 tick time. Around 2000 ticktime, the AIA bought many stocks, and this buyingleads to the market prices increasing. Around 3000tick time, the market prices continued to increase even though the AIA did not bought so many stocks. Here,the fundamental strategy of normal agents in the firstterm of Eq. (1) expected negative return because themarket prices are over the fundamental price. On theother hand, the technical strategy in the second term ofEq. (1) expected larger positive return due to the his-torical positive return around 2000 tick time where theAIA had increased market prices by itself. Therefore,the market prices were able to increase even thoughthe AIA did not bought so many stocks. After then,from around 4000 tick time to around 6000 tick time,the AIA was able to sell stocks with higher prices thanthe prices bought them around 2000 tick time thanksto increasing market prices around 3000 tick time.These trades of the AIA are nothing but market ma-nipulation. This indicates that an artificial intelligencecan discover market manipulation as an optimal in-vestment strategy through learning with an artificialmarket simulation.Fig. 4 shows the time evolution of market pricesand trading volume in the case without the impacts tomarket prices (backtesting) like Fig. 3. Note that Fig.4 has different scale for the vertical axis from those inFig. 2 and Fig. 3. The time evolution of market pricesis exactly same as the case without the AIA becausethe trades of the AIA never impact market prices inFig. 2. Due to lower market prices from the funda-mental price, the AIA tended to buy stocks. Thesetrades of the AIA corresponds to fundamental strat-egy. Thus, in the case of backtesting, the AIA cannotdiscover market manipulation as trading strategy.This indicates possibility that an artificial intelli-gence cannot discover market manipulation throughlearning with backtesting.
In this study, as Fig. 1 shown, I constructed an AItrader using a genetic algorithm that learns in an ar-tificial market simulation, and investigated whetherthe AI trader discovers market manipulation throughlearning despite a builder of the AI trader has no in-tention of market manipulation.As a result, the AI trader discovered market ma-nipulation as an optimal investment strategy. Thisindicates that despite a builder of the AI trader hasno intention of market manipulation, the AI tradercan discover market manipulation as an optimal in-vestment strategy through learning with an artificialmarket simulation in which the AI trader to be ableto automatically learn impacts of its trades to marketprices. On the other hand, this also indicates possi-bility that an AI trader cannot discover market ma-nipulation through learning with backtesting in whichthere are no impacts to market prices.4 T r a d i n g V o l u m e P r i c e Tick TimeTrading Volume of AIA (right)Market Price (left)Fundamental Price (left)
Fig. 3: Time evolution of market prices with the AIA and trading volume (positive and negative number showbuy and sell, respectively) aggregated within each 200 tick time -200204060998010000100201004010060 T r a d i n g V o l u m e P r i c e Tick TimeTrading Volume of AIA (right)Market Price (left)Fundamental Price (left)
Fig. 4: Case without the impacts to market prices(backtesting)This result suggests necessity of regulation, such asobligating builders of artificial intelligence to preventartificial intelligence from performing market manipu-lation.Of course, future works exist. In this study, I sim-ulated eleven situations by one data set of normalagents. In short, I simulated whole my model showed Table 1: Statistics for Returns in the Artificial Marketstandard deviation of returns 0 . . . . . . . ppendix In many previous artificial market studies, the mod-els were verified to see whether they could explainstylized facts, such as a fat-tail or volatility-clustering[2, 4, 5]. A fat-tail means that the kurtosis of pricereturns is positive. Volatility-clustering means thatsquare returns have a positive auto-correlation, andthis auto-correlation slowly decays as its lag becomeslonger. Many empirical studies, e.g., that of Sewell[6], have shown that both stylized facts (fat-tail andvolatility-clustering) exist statistically in almost all fi-nancial markets. Conversely, they also have shownthat only the fat-tail and volatility-clustering are sta-bly observed for any asset and in any period becausefinancial markets are generally unstable.Indeed, the kurtosis of price returns and the auto-correlation of square returns are stably and signifi-cantly positive, but the magnitudes of these valuesare unstable and very different depending on the as-set and/or period. The kurtosis of price returns andthe auto-correlation of square returns were observedto have very broad magnitudes of about 1 ∼
100 andabout 0 ∼ .
2, respectively [6].For the above reasons, an artificial market modelshould replicate these values as significantly positiveand within a reasonable range as I mentioned. It isnot essential for the model to replicate specific valuesof stylized facts because the values of these facts areunstable in actual financial markets.Table 1 lists the statistics, standard deviation of re-turns, kurtosis of price returns, and auto-correlationcoefficient of square returns, where the returns aremeasured within 100 time steps and the statistics areaveraged values of the 100 simulation runs. This tableshows that this model replicated the statistical char-acteristics, fat-tails, and volatility-clustering observedin real financial markets.
Disclaimer
Note that the opinions contained herein are solely those of theauthors and do not necessarily reflect those of SPARX AssetManagement Co., Ltd.
Reference [1] G. Scopino, “Do automated trading systems dream ofmanipulating the price of futures contracts? policingmarkets for improper trading practices by algorithmicrobots,”
Florida Law Review , vol. 67, p. 221, 2016. [Online].Available: https://scholarship.law.ufl.edu/flr/vol67/iss1/5[2] T. Mizuta, “An agent-based model for designing a financialmarket that works well,” arXiv , 2019. [Online]. Available:https://arxiv.org/abs/1906.06000 [3] D. E. Goldberg,
Genetic Algorithms in Search, Optimiza-tion, and Machine Learning . Addison-Wesley Professional,1989.[4] B. LeBaron, “Agent-based computational finance,”
Hand-book of computational economics , vol. 2, pp. 1187–1233, 2006. [Online]. Available: https://doi.org/10.1016/S1574-0021(05)02024-1[5] S.-H. Chen, C.-L. Chang, and Y.-R. Du, “Agent-basedeconomic models and econometrics,”
Knowledge Engineer-ing Review , vol. 27, no. 2, pp. 187–219, 2012. [Online].Available: https://doi.org/10.1017/S0269888912000136[6] M. Sewell, “Characterization of financial time series,”
Research Note, University College London, Department ofComputer Science , no. RN/11/01, 2011. [Online]. Available:https://finance.martinsewell.com/stylized-facts/, no. RN/11/01, 2011. [Online]. Available:https://finance.martinsewell.com/stylized-facts/