Computing trading strategies based on financial sentiment data using evolutionary optimization
CComputing trading strategies based on financial sentimentdata using evolutionary optimization
Ronald HochreiterApril 2015
Abstract
In this paper we apply evolutionary optimization techniques to compute optimal rule-basedtrading strategies based on financial sentiment data. The sentiment data was extracted fromthe social media service StockTwits to accommodate the level of bullishness or bearishnessof the online trading community towards certain stocks. Numerical results for all stocksfrom the Dow Jones Industrial Average (DJIA) index are presented and a comparison toclassical risk-return portfolio selection is provided.
Keywords:
Evolutionary optimization, sentiment analysis, technical trading, portfolio op-timization
In this paper we apply evolutionary optimization techniques to compute optimal rule-basedtrading strategies based on financial sentiment data. The number of application areas in thefield of sentiment analysis is huge, see especially [11] for a comprehensive overview. The field ofFinance attracted research on how to use specific financial sentiment data to find or optimizeinvestment opportunities and strategies, see e.g. [2], [15], and [20].This paper is organized as follows. Section 2 describes the financial sentiment data used forthe evolutionary approach to optimize trading strategies and portfolios. Section 3 presents anevolutionary optimization algorithm to create optimal trading strategies using financial senti-ment data and how to build a portfolio using single-asset trading strategies. Section 4 containsnumerical results obtained with the presented algorithm and a comparison to classical risk-returnportfolio optimization strategies as proposed by [14] using stock market data from all stocks inthe Dow Jones Industrial Average (DJIA) index. Section 5 concludes the paper.
We apply financial sentiment data created by PsychSignal . The PsychSignal technology utilizesthe wisdom of crowds in order to extract meaningful analysis, which is not achievable through thestudy of single individuals, see [12] for a general introduction to measurement of psychologicalstates through verbal behavior. Let a group of individuals together be a crowd. Not all crowdsare wise, however four elements have been identified, which are required to form a wise crowd:diversity of opinion, independence, decentralization and aggregation as proposed by [21]. These a r X i v : . [ q -f i n . P M ] A p r able 1: PsychSignal.com StockTwits sentiment data format per asset.Variable ContentDate Day of the analyzed data. I bull Each message’s language strength of bullishness on a 0-4 scale. I bear Each message’s language strength of bearishness on a 0-4 scale. n bull Total count of bullish sentiment messages. n bear Total count of bearish sentiment messages. n total Total number of messages.four elements are sometimes present in some forms of social media platforms, e.g. in the finan-cial community StockTwits, from which the crowd wisdom used for the evolutionary approachdescribed in this paper is derived.Emotions are regarded as being unique to individual persons and occurring over brief mo-ments in time. Let a mood be a set of emotions together. In order to quantify the collectivemood of a crowd, distinct emotions of individual members within the crowd must be quantified.Subsequently, individual emotions can be aggregated to form a collective crowd mood. PsychSig-nals’ Natural Language Processing Engine is tuned to the social media language of individualtraders and investors based on the general findings of e.g. [9] and of [22] for the financial domain.The engine further targets and extracts emotions and attitudes in that distinct language andcategorizes as well as quantifies these emotions from text. The methodology is based on thelinguistic inquiry and word count (LIWC) project, which is available publicly . See also [16] fora description of an algorithm on how to generate such a semantic lexicon for financial sentimentdata directly.The main idea is to assign a degree of bullishness or bearishness on stocks depending on themessages, which are sent through StockTwits , which utilizes Twitter’s application programminginterface (API) to integrate StockTwits as a social media platform of market news, sentiment andstock-picking tools. StockTwits utilized so called cashtags with the stock ticker symbol, similarto the Twitter hashtag , as a way of indexing people’s thoughts and ideas about companies andtheir respective stocks. The available sentiment data format is described in Tab. 1. The datawas obtained through Quandl , where PsychSignal’s sentiment data for stocks can be accessedeasily.Both intensities I bull and I bear are measured on a real-valued scale from 0 to 4, where 0means no bullish/bearish sentiment and 4 the strongest bullish/bearish sentiment. We normalizethese values to 1 by diving the respective value by 4 and obtain the variables i bull and i bear .Furthermore, we create two relative variables for the number of bullish and bearish messages, i.e. r bull = n bull /n total as well as r bear = n bear /n total , such that we end up in the final data formatwe are going to use for subsequent analysis. See Tab. 2 for an example of the stock with theticker symbol BA (The Boeing Company). We aim at creating an evolutionary optimization approach to generate optimal trading strategiesfor single stocks based on the sentiment analysis data described above. Evolutionary and Ge-netic Programming techniques have been applied to various financial problems successfully. See i bull i bear r bull r bear n total (cid:2) IF( i bull ≥ v ) (cid:3) b (cid:2) AND (cid:3) b & b (cid:2) IF( r bull ≥ v ) (cid:3) b THEN long position . (cid:2) IF( i bear ≥ v ) (cid:3) b (cid:2) AND (cid:3) b & b (cid:2) IF( r bear ≥ v ) (cid:3) b THEN exit position . (1)Each chromosome within the evolutionary optimization process consists of the values( b , b , b , b , v , v , v , v ) , where the b values are binary encoded (0, 1) and the v values are real values between 0 and 1. The b values indicate whether the respective part of the rule notated in square brackets is included(1) or not (0), while the v values represent the concrete values within the conditions. Considerthe following example: the (randomly chosen) chromosome (0,1,1,1,0.4,0.3,0.5,0.2) results in therule-set shown in Eq. (2).IF ( r bull ≥ .
3) THEN long position . IF ( i bear ≥ .
5) AND IF ( r bear ≥ .
2) THEN exit position . (2)In this special case, the sum of b and b as well as b and b must be greater or equal to 1,to have at least one condition for entering and leaving the long position. We end up with ninedifferent possible assignments for b . A repair operator has to be applied after each evolutionaryoperation, which may distort this structure.The evaluation of the chromosomes is such that the respective trading strategy is testedon the in-sample testing set of length T , i.e. we obtain a series of returns r , . . . , r T for eachchromosome, which can be evaluated with different financial metrics. The following strategyperformance characteristics are considered: • The cumulative return r , and the standard deviation σ .3able 3: Statistical summary of sentiment values for stock BA 2010-2014.Minimum First Quantile Median Mean Third Quantile Maximum i bull i bear • The maximum drawdown d , and the Value-at-Risk v α ( α = 0 . • the ratio s of expected return divided by the standard deviation, which is based on theSharpe-ratio proposed by [19].We use simple mutation operators for new populations because the chromosome encodingof the investment rule described above is short, i.e. contains only eight genes. The followingmutation operators are applied: • b binary flip: One randomly selected gene of the binary b part is 0 − • v random mutation: One randomly selected gene of the binary v part is replaced by auniform random variable between 0 and 1. • v mutation divided in half: One randomly selected gene of the binary v part is divided inhalf. The rationale of this operation is that the intensities of bullishness and bearishnessare often small, see e.g. Tab. 3 for the statistics of the sentiment values for a selected stock.Besides these operators, elitist selection is applied as well as a number of random additionswill be added to each new population. The structure of the algorithm is a general geneticalgorithm, see e.g. [1] for a description of this class of meta-heuristics.The analysis above is based on single assets. To compose a portfolio out of the single invest-ment strategies, the resulting portfolio will be created as an equally weighted representation ofall assets, which are currently selected to be in a long position by its respective trading strategyfor each day. In this section we begin with a description of the data used to compute numerical results inSection 4.1. Section 4.2 summarizes the in-sample and out-of-sample results of the evolutionarysentiment trading strategy. A short overview of classical risk-return portfolio optimization isgiven in Section 4.3, and finally a performance comparison is presented in Section 4.4. Everythingwas implemented using the statistical computing language R [18].
We use data from all stocks from the Dow Jones Industrial Average (DJIA) index using thecomposition of September 20, 2013, i.e. using the stocks with the ticker symbols AXP, BA, CAT,CSCO, CVX, DD, DIS, GE, GS, HD, IBM, INTC, JNJ, JPM, KO, MCD, MMM, MRK, MSFT,NKE, PFE, PG, T, TRV, UNH, UTX, V, VZ, WMT, XOM.Training data is taken from the beginning of 2010 until the end of 2013. The out-of-sampletests are applied to data from the year 2014. 4 .2 Results of the Evolutionary Optimization
For each stock, the optimal strategy was computed. The evolutionary parameters were set to beas follows: • The initial population size has been set to 100, and each new population • contains the 10 best chromosomes of the previous population (elitist selection), as well as •
20 of each of the three mutation operators described above, and •
10 random chromosomes, such that the population size is 80.For evaluation purposes, the parameter s will be maximized. Of course, the system is flexibleto use any other risk metric or a combination of metrics. See Tab. 4 for the in-sample performanceresults comparing a long-only buy-and-hold strategy of each asset compared to the trading strat-egy of the best respective strategy, e.g. the best strategy for AXP is (1 , , , , . , . , . , . , , , , . , . , . , . , , , , . , . , . , .
24) togive an impression of single strategy results. The cumulative return performance r is raised(sometimes significantly) for almost all assets except for MCD, UTX, V. However, in those threecases the decrease in profit is low. The standard deviation σ is lower (i.e. better) in all cases,which was expected as the algorithm leaves the long-position for a certain time, such that thestandard deviation clearly has to decrease. The Sharpe-ratio like metric s is better for all assetsbut DIS, JNJ, UTX, XOM. Again, the loss in all four cases is low compared to the gain of theother positions. In summary, the in-sample results show that the fitting of the algorithm worksvery well. To compare the performance of the portfolio created with single asset investment strategiesbased on financial sentiments with a standard approach to portfolio optimization, we construct aportfolio using classical risk-return portfolio selection techniques. [14] pioneered the idea of risk-return optimal portfolios using the standard deviation of the portfolios profit and loss functionas risk measure. In this case, the optimal portfolio x is computed by solving the quadraticoptimization problem shown in Eq. 3. The investor needs to estimate a vector of expectedreturns r of the assets under consideration as well as the covariance matrix C . Finally theminimum return target µ has to be defined. Any standard quadratic programming solver can beused to solve this problem numerically.minimize x T C x subject to r × x ≥ µ (cid:80) x = 1 (3)In addition, we also compare the performance to the 1-over-N portfolio, which equally weightsevery asset under consideration. It has been shown that there are cases, where this simple strategyoutperforms clever optimization strategies, see e.g. [10]. The asset composition of the optimal Markowitz portfolio is shown in Tab. 5 - only eight out ofthe 30 assets are selected. The underlying covariance matrix was estimated from daily returnsof the training data, i.e. using historical returns from the beginning of 2010 until the end of5able 4: Single stock in-sample results of the evolutionary optimization.Long-only stock Trading strategy r σ s r σ s
AXP 1.223 0.016 0.057 1.574 0.013 0.068BA 1.450 0.016 0.063 1.771 0.013 0.070CAT 0.575 0.018 0.034 0.978 0.014 0.043CSCO -0.070 0.019 0.006 1.246 0.011 0.061CVX 0.597 0.013 0.042 0.723 0.012 0.044DD 0.912 0.015 0.051 1.177 0.011 0.070DIS 1.351 0.015 0.065 1.522 0.013 0.065GE 0.842 0.015 0.048 1.054 0.013 0.050GS 0.042 0.019 0.012 0.662 0.010 0.041HD 1.825 0.014 0.082 2.209 0.012 0.087IBM 0.430 0.012 0.036 0.711 0.005 0.083INTC 0.249 0.015 0.022 0.600 0.009 0.043JNJ 0.415 0.008 0.045 0.415 0.008 0.041JPM 0.399 0.019 0.027 0.873 0.016 0.037KO -0.277 0.019 -0.004 0.363 0.006 0.042MCD 0.549 0.009 0.052 0.515 0.006 0.060MMM 0.688 0.013 0.048 0.795 0.012 0.051MRK 0.359 0.012 0.032 0.516 0.010 0.041MSFT 0.222 0.014 0.021 0.509 0.012 0.031NKE 0.190 0.022 0.022 0.511 0.019 0.030PFE 0.677 0.012 0.048 0.805 0.011 0.049PG 0.332 0.009 0.036 0.406 0.007 0.046T 0.238 0.010 0.026 0.397 0.007 0.040TRV 0.805 0.012 0.053 0.946 0.011 0.064UNH 1.400 0.016 0.063 2.443 0.013 0.091UTX 0.621 0.013 0.042 0.563 0.011 0.042V 1.530 0.017 0.062 1.494 0.010 0.072VZ 0.471 0.011 0.041 0.572 0.009 0.045WMT 0.464 0.009 0.045 0.654 0.007 0.058XOM 0.473 0.012 0.039 0.506 0.010 0.0366able 5: Optimal Markowitz portfolio using daily return data from 2010-2013.Ticker symbol HD JNJ MCD PG UNH V VZ WMTPortfolio weight [%] 10.26 16.69 22.67 11.92 6.41 4.22 7.56 20.27Table 6: Selected risk metrics for the different out-of-sample tests.Markowitz 1-over-N EvolutionarySemi Deviation 0.0042 0.0048 0.0038Downside Deviation (Rf=0%) 0.0040 0.0047 0.0037Maximum Drawdown 0.0631 0.0687 0.0549Historical VaR (95%) -0.0092 -0.0105 -0.0081Historical ES (95%) -0.0125 -0.0157 -0.01242013. This portfolio is used as a buy-and-hold portfolio over the year 2014. This out-of-sampleperformance is shown in Fig. 1 . While the performance of the 1-over-N portfolio is not showngraphically, Fig. 2 depicts the performance of a portfolio, which is created by equally weightingall single asset trading strategies computed by the evolutionary optimization algorithm based onfinancial sentiment data into one portfolio. To get a better impression of the differences see Tab.6, where some important risk metrics are summarized for all three strategies. The evolutionarytrading portfolio exhibits better risk properties than both other portfolios in all five metrics.Especially important is the reduction of the maximum drawdown, which is of importance toasset managers nowadays, because investors are increasingly looking to this metric if they aresearching for secure portfolios. In this paper an evolutionary optimization approach to compute optimal rule-based tradingstrategies based on financial sentiment data has been developed. It can be shown that a port-folio composed out of the single trading strategies outperforms classical risk-return portfoliooptimization approaches in this setting. The next step is to include transaction costs to see howthis active evolutionary strategy loses performance when transaction costs are considered. Futureextensions include extensive numerical studies on other indices as well as using and comparingdifferent evaluation risk metrics or a combination of metrics. One may also consider to create amore flexible rule-generating algorithm e.g. by using genetic programming. Finally, to achievean even better out-of-sample performance the recalibrating of the trading strategy can be doneusing a rolling horizon approach every month.
References [1] C. Blum and A. Roli. Metaheuristics in combinatorial optimization: Overview and concep-tual comparison.
ACM Computing Surveys , 35(3):268–308, 2003.[2] J. Bollen, H. Mao, and X. Zeng. Twitter mood predicts the stock market.
Journal ofComputational Science , 2(1):1–8, 2011. Performance graphs are generated using the
PerformanceAnalytics
R package [17]. . . . . C u m u l a t i v e R e t u r n Performance − . − . . . . D a il y R e t u r n Jan 02 Feb 03 Mar 03 Apr 01 May 01 Jun 02 Jul 01 Aug 01 Sep 02 Oct 01 Nov 03 Dec 01 Dec 31 − . − . − . . D r a w do w n Figure 1: Out-of-sample performance of a buy-and-hold Markowitz portfolio in 2014.8 . − . − . . . . . . C u m u l a t i v e R e t u r n Performance − . − . . . . D a il y R e t u r n Jan 02 Feb 03 Mar 01 Apr 01 May 01 Jun 02 Jul 01 Aug 01 Sep 02 Oct 01 Nov 03 Dec 01 Dec 31 − . − . − . D r a w do w n Figure 2: Out-of-sample performance of an equally weighted portfolio out of the evolutionarysentiment trading strategies in 2014. 93] A. Brabazon and M. O’Neill, editors.
Natural Computing in Computational Finance , volume100 of
Studies in Computational Intelligence . Springer, 2008.[4] A. Brabazon and M. O’Neill, editors.
Natural Computing in Computational Finance, Volume2 , volume 185 of
Studies in Computational Intelligence . Springer, 2009.[5] A. Brabazon and M. O’Neill. Evolving technical trading rules for spot foreign-exchangemarkets using grammatical evolution.
Computational Management Science , 1(3-4):311–327,2004.[6] A. Brabazon and M. O’Neill. Intra-day trading using grammatical evolution. In A. Brabazonand M. O’Neill, editors,
Biologically Inspired Algorithms for Financial Modelling , pages 203–210. Springer, 2006.[7] A. Brabazon, M. O’Neill, and D. Maringer, editors.
Natural Computing in ComputationalFinance, Volume 3 , volume 293 of
Studies in Computational Intelligence . Springer, 2010.[8] R. G. Bradley, A. Brabazon, and M. O’Neill. Evolving trading rule-based policies.
LectureNotes in Computer Science , 6025:251–260, 2010.[9] S. R. Das and M. Y. Chen. Yahoo! for Amazon: Sentiment extraction from small talk onthe web.
Management Science , 53(9):1375–1388, 2007.[10] V. DeMiguel, L. Garlappi, and R. Uppal. Optimal versus naive diversification: How ineffi-cient is the 1/n portfolio strategy?
Review of Financial Studies , 22(5):1915–1953, 2009.[11] R. Feldman. Techniques and applications for sentiment analysis.
Communications of theACM , 56(4):82–89, 2013.[12] L. A. Gottschalk and G. C. Gleser.
Measurement of Psychological States Through the Con-tent Analysis of Verbal Behaviour . University of California Press, 1969.[13] P. Lipinski and J. J. Korczak. Performance measures in an evolutionary stock trading expertsystem.
Lecture Notes in Computer Science , 3039:835–842, 2004.[14] H. Markowitz. Portfolio selection.
The Journal of Finance , 7(1):77–91, 1952.[15] N. Oliveira, P. Cortez, and N. Areal. On the predictability of stock market behavior usingStockTwits sentiment and posting volume.
Lecture Notes in Computer Science , 8154:355–365, 2013.[16] N. Oliveira, P. Cortez, and N. Areal. Automatic creation of stock market lexicons forsentiment analysis using StockTwits data. In
Proceedings of the 18th International DatabaseEngineering & Applications Symposium , pages 115–123. ACM, 2014.[17] B. G. Peterson and P. Carl.
PerformanceAnalytics: Econometric tools for performance andrisk analysis , 2014. URL http://CRAN.R-project.org/package=PerformanceAnalytics .R package version 1.4.3541.[18] R Core Team.
R: A Language and Environment for Statistical Computing . R Foundationfor Statistical Computing, Vienna, Austria, 2014. URL .[19] W. F. Sharpe. The sharpe ratio.
The Journal of Portfolio Management , 21(1):49–58, 1994.[20] J. Smailovi´c, M. Grˇcar, N. Lavraˇc, and M. ˇZnidarˇsiˇc. Stream-based active learning forsentiment analysis in the financial domain.
Information Sciences , 285:181–203, 2014.1021] J. Surowiecki.
The Wisdom of Crowds . Anchor Books, 2005.[22] R. Tumarkin and R. F. Whitelaw. News or noise? Internet postings and stock prices.