AAdversarial trading
Alexandre MiotJanuary 11, 2021
Abstract
Adversarial samples have drawn a lot of attentionfrom the Machine Learning community in the pastfew years. An adverse sample is an artificial datapoint coming from an imperceptible modification ofa sample point aiming at misleading. Surprisingly,in financial research, little has been done in rela-tion to this topic from a concrete trading point ofview. We show that those adversarial samples canbe implemented in a trading environment and havea negative impact on certain market participants.This could have far reaching implications for finan-cial markets either from a trading or a regulatorypoint of view.
Keywords—
Adversarial Samples - Machine Learning- Algorithmic Trading
Adversarial samples have received a lot of attention re-cently in the Machine Learning (ML) community. Wefocus here on adverse samples in a trading context andon a market toy model. From (Kurakin et al., 2018),an adversarial sample is “a sample of input data whichhas been modified very slightly in a way that is in-tended to cause a machine learning classifier to misclas-sify it”. While market manipulation is usually basedon a handful of well known techniques, see (Putnins,2018), one’s could harness the recent progress of MLon adversarial samples to build more elaborate tech-niques, see (Wiyatno, Xu, Dia, & de Berker, 2019) or(Machado, Silva, & Goldschmidt, 2020) for a review.We call adversarial trading the use of adversarial sam-ples to trade. Financial markets are especially exposeddue to the high quantity of data and the pervasivenessof algorithmic trading. On a simulated order book, weinvestigate the possible implications of adversarial trad-ing.
Since the seminal work of (Dalvi, Domingos, Mausam,Sanghai, & Verma, 2004), (Lowd & Meek, 2005) and themore recent (Szegedy et al., 2013), adversarial samplesis an active field of research in Machine Learning. It hasbeen explored in computer vision (Moosavi-Dezfooli,Fawzi, & Frossard, 2015), human vision (Elsayed etal., 2018), real 3D objects (Athalye, Engstrom, Ilyas,& Kwok, 2017), tabular data (Ballet et al., 2019),time series (Mode & Hoque, 2020), speech (Carlini& Wagner, 2018) or from a theoretical point of viewin (Dohmatob, 2019), (Tsipras, Santurkar, Engstrom,Turner, & Madry, 2018), or (Fawzi, Fawzi, & Frossard,2017). See (Wiyatno et al., 2019) for a review. Yet,even if (Faghan, Piazza, Behzadan, & Fathi, 2020) hasa similar point of view, to our knowledge little has beendone to explore the implications of adversarial samplesin financial markets from a practical trading perspec-tive.
To measure the efficiency of our adversarial samples, wefirst build a market simulator. We then build a datasetand train a classifier predicting the direction of the nextmarket move. From adverse samples, we finally buildan adversarial trading agent and show its impact onother agents.
We build a simple matching engine to simulate a centrallimit order book. A trading round is composed of twoconsecutive steps:1. no more book orders can be fulfilled. All marketparticipants determine which quotes they want tosend and which unmatched quotes they want tocancel a r X i v : . [ q -f i n . T R ] D ec . they place their quotes and cancels into the book:orders are executed partially or fully as soon asthey match.At the end of the second step, all matching orders havebeen executed and we are back to the first step. Asimulation is a set of trading rounds for a set of agents.In the following, we will note b jt and o jt , the bid andoffer levels at the j th depth and trading round t . qb jt and qo jt are the respective quantities. A typical orderbook will be : Bids OffersLevel Quantity Level Quantity b t qb t o t qo t b t qb t o t qo t o t qo t Market participants are named agents. Four types ofagents are defined: investor agents have predetermined and constantbearish or bullish view called bias i.e. a bullishagent will only buy the market and a bearish oneonly sell. At each round, they cancel any previ-ous unmatched order and place a buy or sell orderaccording to their bias market maker agents place a buy and a sell orderfor the same size at each round cancelling any pre-vious unmatched order noisy agent place random buy and sell orders at eachround adversarial agent place adverse orders at eachround.In order to have a balanced market, for each simula-tion, the number of bullish and bearish investor agentsis roughly the same. Each agent has a stop loss. Ifbreached then all its unmatched orders are cancelledand it will not be able to trade any more.
Before placing orders, a reference price is computedfrom the market book. This is essentially the mid mar-ket price but restricting price jumps: bm t = 12 (cid:0) b t + o t (cid:1)(cid:103) bm t = (cid:40) bm t if (cid:12)(cid:12)(cid:12) bm t n (cid:80) ni =1 m t − i − (cid:12)(cid:12)(cid:12) < αm t − otherwise m t = m t − min (cid:32) α, max (cid:32) − α, (cid:103) bm t m t − (cid:33)(cid:33) , in our simulations average depth is around 17 where m t is the reference price at trading round t , b t and o t being the best bid and offer at t , n = 10 and α = 10%. Bearish (respectively bullish) investor agentsplace a sell (respectively buy) order randomly aroundthis reference price plus (respectively minus) one per-cent.Market maker agents use the best bid and offer butskew their quotes depending on the market imbalance.If the market has more quantity on the bid (respec-tively on the offer) side then the market maker will bid(respectively offer) more aggressively. Noting mb t and mo t the bid and offer levels of the market maker: i = n (cid:88) j =1 qb jt − m (cid:88) j =1 qo jt ι = i> − i< mb t = b t × . × (1 + ια ) mo t = o t × . × (1 + ια ) .Noisy or adversarial agents might place orders at anydepth in the book either randomly for the noisy agent oraccording to an adversarial strategy for the adversarialagent. Adverse agents place orders which are designed to havean adverse effect on other market participants. In ourframework, we know that modifying the market imbal-ance might impact market maker agents. The adverseagent, though, has no access to the market maker strat-egy. For all agents, only the anonymous order book isknown. We build a surrogate model: a classifier whichwill try to predict if the market will move up or down atthe next trading round. We use a logistic regression onfeatures X t being the concatenation of bid levels, bidquantities, offer levels and offer quantities. The orderbook can have any depth which can vary with time. Wechoose a constant depth of 20 adding missing values of-1 for prices and 0 for quantities if the book depth issmaller than 20: X t = ( b t , b t , − , . . . , qb t , qb t , , . . . ,o t , o t , o t , − , . . . , qo t , qo t , qo t , . . . ) ,as a result X t ∈ R . The surrogate model has twoimportant prerequisites:1. an agent must be able to have an influence on thefeatures2. the predicted quantity must be a relevant tradingfeature to the targetted agents. n test data our surrogate model has a precisionaround 70%. We then build adversarial examples using FGSM asdescribed in (Goodfellow, Shlens, & Szegedy, 2014).We choose samples correctly predicted by the surrogatemodel and build for each sample an adverse perturba-tion defined as δ x = argmin f ( x + r ) (cid:54) = f ( x ) || r || ,and approximated in the FGSM method as δ x = (cid:15) ∇ r L ( x + r, f ( x )) ,where f is the surrogate classifier, x a correctly clas-sified sample, (cid:15) > L is theclassifier cost . Supposing that we impose (cid:107) r (cid:107) ∞ < (cid:15) ,we want to maximize the loss:argmax (cid:107) r (cid:107) ∞ ≤ (cid:15) L ( x + x ) − L ( r ) ∼ argmax (cid:107) r (cid:107) ∞ ≤ (cid:15) r · ∇ L = (cid:15) sign ∇ L .Our adversarial sample is fast and simple but otherconstraints might be needed. For example, imaginethat the perturbation is negative for the offer quan-tity at the j th depth: it would mean that we shouldplace orders to decrease this quantity. This is impossi-ble in practice without massively impacting the book.One way to handle constraints on adversarial perturba-tions would be to use techniques described in (Carlini& Wagner, 2016). Another possibility would have beento use Adversarial Transformation Networks of (Baluja& Fischer, 2017). In section 3.4.3 we describe how theadversarial agent deals with possibly unrealistic pertur-bations.It is important to note that noise and adversarial per-turbations are very different. On figure 1, we plot theaccuracy of initially correctly predicted samples as weincrease the amplitude of perturbation for both noisyand adversarial perturbations. Noise is detrimental topredictions but far less than adversarial samples for agiven (cid:15) bound.In a more general setting, building adversarial samplesmight be slow. We build a map x (cid:55)→ δ x mapping asample to an approximate perturbation. We train amultiple target classifier to predict the signs of the gra-dient η x = sign ∇ L x . A random forest classifier givesvery good results. This classifier is named adversarialestimator. here the negative log-likelihood of the logistic regression Figure 1: Accuracy loss as the amplitude of the per-turbation (cid:15) increases. Adversarial perturbations (red)are much more effective than adding noise (blue) of thesame amplitude.
The final step is to build an adversarial agent. The taskof mapping an adverse perturbation in feature spaceinto actual orders is not obvious. As we have seen, theperturbation might indicate to lower the price at thethird depth or decrease the quantity at the fifth depth,for example. From our adversarial estimator we retrievea vector η x ∈ {− , } given by the previously trainedmulti-target random forest classifier: η x = (1 , , . . . (cid:124) (cid:123)(cid:122) (cid:125) bid price i =1 ... , − , , . . . (cid:124) (cid:123)(cid:122) (cid:125) bid quantities i =20 ... , − , − , . . . (cid:124) (cid:123)(cid:122) (cid:125) offer prices i =40 ... , − , , . . . (cid:124) (cid:123)(cid:122) (cid:125) offer quantities i =60 ... ) .The adversarial agent discards information given byprices i.e. it discards the values η ix where i = 1 , . . . , i = 40 , . . . ,
60. For i ∈ (cid:74) , (cid:75) , if the sign of thebid quantity η ix is +1 then a bid order at this depthlevel is added for 1 lot (see table 1 for averages sizesin the book). If the sign is -1 then an offer order atthe corresponding depth is placed at the offer price forthis depth. The same thing is done for i ∈ (cid:74) , (cid:75) . Allprevious orders from previous trading round are can-celled. The noisy agent works exactly the same wayexcept that instead of getting η x ∈ {− , } from theadverse estimator it draws them randomly. Bid OfferFirst depth ∼ ∼ ∼ ∼ ∼ ∼ ∈ [10 , ∈ [10 , ∈ [4 , ∈ [4 , Table 1: Average book quantities up to 20 th depth. Results
We run 10 rounds of 500 simulations each comprising200 trading rounds for each of the following setups:- 40 investor agents and one market maker agent- 40 investor agents, one market maker agent andone noisy agent- 40 investor agents, one market maker agent andone adversarial agent.It gives us a total of 10 × ×
200 = 1 million tradingrounds for each of the three setups. Simulated trajecto-ries are not too dissimilar from financial time series asseen in figure 2 and are good enough for our purpose.The market maker agent takes into account the mar-Figure 2: Randomly selected time series of bid (dashred) and offer (point and dash blue) of the first depthof simulated books.ket imbalance when quoting. As a result, an adversarymight be able to impair the market maker profit andloss (P&L) by placing small orders in the book. Thisis what we see from our simulations in table 2 lookingat the cumulative P&L of the market maker agent un-der our three setups. We see that the market maker’sperformance deteriorates when an adversarial or noisyagent is in the market. Consistently, for all simulations:- the median P&L of the market maker agent ishigher without noisy or adverse agent- the median P&L of the market maker agent ishigher with noisy agent than with adverse agent.Yet, the difference between the noisy and adverse agentsetups can be small. Also, the adversarial agent doesnot manage to benefit from fooling the market makeras its average P&L is always negative around -50,000.Actually, the investor agents are benefiting from theadverse agent with an average P&L going from -4,000without adverse agent to -2,000 with an adverse agent.As a result, if one’s wanted to build a profitable adver-sarial agent more work is necessary. Yet, this is not ouraim here and we think that these results demonstratethat adversarial trading can work in a concrete tradingsetup.
Mean Median 1 st quartile 3 rd quartileWithoutadversary 177,000 54,000 39,000 199,000With noisyagent 71 % 95 % 80 % 70 %With adver-sary agent 67 % 92 % 72 % 70 % Table 2: Rounded P&L statistics of the market makeragent under different setups (in percentage of P&Lwithout adversary for adversarial setups).
We have seen that on a simulated market, we can de-sign adversarial agents which actions, though difficultto notice, can have a negative impact on other agents.A legitimate question is to ask if this can be transposedinto real markets. To answer this question two distinctaspects have to be taken into consideration:1. the ability to find adversarial samples2. the ability to find adversarial agents able to “im-plement” these adverse samples.On the first point, even if research is still very ac-tive, several theoretical articles point to a similar di-rection involving that it is probably easier in real con-ditions. The most relevant theoretical work to ourtopic is (Fawzi, Fawzi, & Fawzi, 2018): any classifieron a feature space which can be approximated by ahighly dimensional generative model is prone to ad-verse attacks. Latest progress in market generators,see (Wiese, Knobloch, Korn, & Kretschmer, 2020),(Koshiyama, Firoozye, & Treleaven, 2020) or (B¨uhler,Horvath, Lyons, Arribas, & Wood, 2020) tend to showthat market prices can be approximated by generativemodels. Moreover, a more complex classifier distin-guishing more market conditions is even less robust,the robustness decreasing with the number K of classes.Also, under mild assumptions, transferability of adver-sarial samples is granted blurring the distinction be-tween white box and black box attacks. This entailsthat training a surrogate model might have high prob-ability of working if the attacked model has low gener-alization error .On the second point, to our knowledge little researchhas been carried out. We think though that our findingsshow that they might exist in practice. As a result, reg-ulators might want to tackle the subject, knowing thatin complex markets tracking these adversarial agentsmight well be difficult, see (Wang et al., 2020) though. which is a precondition to have an algorithmic model inproduction in the first place Conclusion
In this article, we found that in a simulated frameworkit is possible to use adversarial samples and implementthem as a trading strategy in order to negatively impactsome market participants. While finding adversarialsamples is a relatively easy task, implementing them ina trading environment is more difficult but as we haveseen possible. Importantly, regulators might want toscrutinize application of adversarial trading to insuremarket integrity. Finally, one could ask if adversarialsamples do apply to human brain too. Empirically, itdoes not seem to be the case and it might also be truein a trading environment: human traders might be lessefficient than machines but way sturdier.
The author report no conflicts of interest. The authoralone is responsible for the content and writing of thepaper. eferences Athalye, A., Engstrom, L., Ilyas, A., & Kwok, K.(2017). Synthesizing robust adversarial ex-amples.
ArXiv , .Ballet, V., Renard, X., Aigrain, J., Laugel, T.,Frossard, P., & Detyniecki, M. (2019). Imper-ceptible adversarial attacks on tabular data. arXiv , . (NeurIPS 2019 Work-shop on Robust AI in Financial Services:Data, Fairness, Explainability, Trustworthi-ness and Privacy (Robust AI in FS 2019),)Baluja, S., & Fischer, I. (2017). Adversarial trans-formation networks: Learning to generate ad-versarial examples. CoRR , .B¨uhler, H., Horvath, B., Lyons, T., Arribas, I. P.,& Wood, B. (2020). A data-driven marketsimulator for small data environments. arXiv , .Carlini, N., & Wagner, D. A. (2016). Towardsevaluating the robustness of neural networks. ArXiv , .Carlini, N., & Wagner, D. A. (2018). Audio adver-sarial examples: Targeted attacks on speech-to-text. CoRR , .Dalvi, N., Domingos, P., Mausam, Sanghai, S., &Verma, D. (2004). Adversarial classification.In Proceedings of the tenth acm sigkdd in-ternational conference on knowledge discov-ery and data mining (p. 99–108). New York,NY, USA: Association for Computing Ma-chinery.Dohmatob, E. (2019). Generalized no freelunch theorem for adversarial robustness. InK. Chaudhuri & R. Salakhutdinov (Eds.),
Proceedings of the 36th international con-ference on machine learning (Vol. 97, pp.1646–1654). Long Beach, California, USA:PMLR.Elsayed, G., Shankar, S., Cheung, B., Papernot,N., Kurakin, A., Goodfellow, I., & Sohl-Dickstein, J. (2018). Adversarial examplesthat fool both computer vision and time-limited humans. In S. Bengio, H. Wallach,H. Larochelle, K. Grauman, N. Cesa-Bianchi,& R. Garnett (Eds.),
Advances in neural in-formation processing systems (Vol. 31, pp.3910–3920). Curran Associates, Inc.Faghan, Y., Piazza, N., Behzadan, V., & Fathi,A. (2020, 10). Adversarial attacks on deep algorithmic trading policies.
ArXiv , .Fawzi, A., Fawzi, H., & Fawzi, O. (2018). Adver-sarial vulnerability for any classifier. ArXiv .Fawzi, A., Fawzi, O., & Frossard, P. (2017, Aug).Analysis of classifiers’ robustness to adversar-ial perturbations.
Machine Learning , (3),481–508.Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014).Explaining and harnessing adversarial exam-ples. Proceedings, International Conferenceon Learning Representations .Koshiyama, A., Firoozye, N., & Treleaven, P.(2020). Generative adversarial networks forfinancial trading strategies fine-tuning andcombination.
Quantitative Finance , 1–17.Kurakin, A., Goodfellow, I. J., Bengio, S., Dong,Y., Liao, F., Liang, M., . . . Abe, M. (2018).Adversarial attacks and defences competi-tion.
ArXiv , .Lowd, D., & Meek, C. (2005). Adversarial learn-ing. In Proceedings of the eleventh acm sigkddinternational conference on knowledge discov-ery in data mining (p. 641–647). New York,NY, USA: Association for Computing Ma-chinery.Machado, G., Silva, E., & Goldschmidt, R. R.(2020). Adversarial machine learning in im-age classification: A survey towards the de-fender’s perspective.
ArXiv , .Mode, G. R., & Hoque, K. A. (2020). Adversar-ial examples in deep learning for multivari-ate time series regression. .Moosavi-Dezfooli, S., Fawzi, A., & Frossard, P.(2015). Deepfool: a simple and accuratemethod to fool deep neural networks. In Proceedings of ieee conference on computervision and pattern recognition (cvpr), 2016 (Vol. abs/1511.04599).Putnins, T. J. (2018). An overview of market ma-nipulation. In
Handbook of corruption andfraud in financial markets: Malpractice, mis-conduct and manipulation.
Wiley.Szegedy, C., Zaremba, W., Sutskever, I., Bruna,J., Erhan, D., Goodfellow, I., & Fergus, R.(2013). Intriguing properties of neural net-works.
ArXiv , .Tsipras, D., Santurkar, S., Engstrom, L., Turner,6., & Madry, A. (2018). Robustness may beat odds with accuracy. ArXiv , .Wang, S., Nepal, S., Grobler, M., Rudolph, C.,Chen, T., & Chen, S. (2020). Adversarial de-fense by latent style transformations. arXiv , .Wiese, M., Knobloch, R., Korn, R., & Kretschmer,P. (2020). Quant gans: deep generation offinancial time series. Quantitative Finance , (9), 1419-1440.Wiyatno, R. R., Xu, A., Dia, O., & de Berker,A. (2019). Adversarial examples in mod-ern machine learning: A review. ArXiv ,1911.05268