A Deep Learning Approach for Dynamic Balance Sheet Stress Testing
Anastasios Petropoulos, Vassilis Siakoulis, Konstantinos P. Panousis, Theodoros Christophides, Sotirios Chatzis
AA Deep Learning Approach for Dynamic Balance SheetStress Testing
Anastasios Petropoulos ∗ , Vassilis Siakoulis, Konstantinos P. Panousis,Theodoros Christophides, and Sotirios Chatzis Department of Electrical Engineering, Computer Engineering and InformaticsCyprus University of Technology, Limassol, Cyprus
Abstract
In the aftermath of the financial crisis, supervisory authorities have considerablyimproved their approaches in performing financial stress testing. However, theyhave received significant criticism by the market participants due to the method-ological assumptions and simplifications employed, which are considered as notaccurately reflecting real conditions. First and foremost, current stress testingmethodologies attempt to simulate the risks underlying a financial institutionsbalance sheet by using several satellite models, making their integration a reallychallenging task with significant estimation errors. Secondly, they still sufferfrom not employing advanced statistical techniques, like machine learning, whichcapture better the nonlinear nature of adverse shocks. Finally, the static balancesheet assumption, that is often employed, implies that the management of abank passively monitors the realization of the adverse scenario, but does nothingto mitigate its impact. To address the above mentioned criticism, we introducein this study a novel approach utilizing deep learning approach for dynamicbalance sheet stress testing. Experimental results give strong evidence that deeplearning applied in big financial/supervisory datasets create a state of the artparadigm, which is capable of simulating real world scenarios in a more efficientway. ∗ Corresponding author
Email address: [email protected] (Anastasios Petropoulos)
Preprint submitted to Journal of Forecasting September 24, 2020 a r X i v : . [ q -f i n . C P ] S e p eywords: Stress Testing, Deep Learning, Bayesian Model Averaging, CapitalAdequacy Ratio, Forecasting, Neural Networks, Dynamic balance sheet,Constant balance sheet assumption
1. Introduction & Motivation
Financial Stability is a core component for economic prosperity of countriesand individuals. The recent financial crises had significant impact in the life ofmany individuals across the globe through the realization of significant incomereduction, rising unemployment and economic slowdown [1]. The recent expe-rience from methods of risk management employed so far shows that they failto provide early warnings to central governments and central banks in order toproactively intervene and prevent such adverse financial events. Banks, regu-latory authorities, and international organizations (like IMF) performed stresstesting exercises long before the financial crisis of 2007. Nevertheless, stresstesting exercises before Lehmans default failed to predict the unprecedentedeconomic turmoil, because they disregard the propagation channels of a defaultevent through the whole micro macro dynamics of the global interconnectedfinancial system. Additionally, the non-linear relationships that realised betweenthe macro economy and the financial balance sheets was not adequately captureddue to the broadly use of linear regression models. Weaknesses also in thevalidation function of stress testing frameworks also decreased the confidencein the quantification of the impact an adverse scenario in the banking system.Since then, market participants and regulators have performed rigorous stresstesting exercises by expanding the scenarios to be assessed, using more granulardata, and in some cases attempting to quantify second round effects stemmingfrom a liquidity shock or from the default of a counterparty.Another aspect in the current regime, i.e. post crisis, of supervisory regulationis the collection of a significant amount of granular information as a responseto a more proactive supervision. Although, the creation of big data sets isthe new era in banking supervision, regulators havent yet explored statistical2echniques from other fields, like machine learning, to extract more informationregarding the risks in their banking systems. Segmentation, classification anddata mining functionalities are important tools for regulators to spot weaknessesin the supervised financial entities, which can be further enhanced by usingmachine learning techniques.Machine learning algorithms has dramatically improved the capabilities ofperforming pattern recognition (like speech recognition, image recognition) andforecasting, so that they offer state of the art performance in various scientificfields like biology, engineering etc. Their structure offers the ability to adjustin streaming sequences using continuous learning algorithms, and recognizenew and evolving patterns in time series data. In addition, deep learning isproven to effectively deal with high dimensional data. Recent studies suggestthat due to their complicated and non-parametric structure, machine learningtechniques could lead to better predictive performance in financial time seriesmodelling problems [2, 3, 4]. This can be attributed to their capacity to learn andadapt to new data. Thus, improve their performance over time, offer increasedcapabilities to capture non-linear relationships, and decompose the noise thatoften exist in financial data. Furthermore, this new generation of statisticalalgorithms offer the necessary flexibility in modelling multivariate time series, asits structure includes a cascade of many layers with non-linear processing agents.Deep learning networks base their functionality in the interaction of layers thatsimulate the abstraction and composition functionalities of the human brain.Therefore, via capturing the full spectrum of information contained in financialdatasets, they are capable of exploring in depth the inherent complexity of theunderlying dynamics in big and high dimensional time series data.Motivated by the recent trends in the machine learning literature, thisempirical study introduces a new statistical technique for stress testing usingdeep learning algorithms to model banks financial data in a holistic way. Inparticular, shocks are propagated to banks’ balance sheets by simultaneouslytraining deep neural networks with macro and financial variables, thus, takingadvantage of their capabilities to capture more information hidden in big datasets.3e develop inference algorithms for our networks, suitable for learning financialtime series data on a multivariate forecasting setup.The main contribution of this study is that it proposes a holistic frameworkfor balance sheet stress testing, which overcomes the limitations of the currentlyapproaches and yields more robust and close to reality results by loosening thestatic balance sheet assumption. Our research analysis lies at the intersection ofcomputational finance and statistical machine learning, leveraging the uniqueproperties and capabilities of deep learning networks to increase the predictionefficacy and minimizing the modelling error. Under the proposed approach, fore-casting of balance sheet items can be heavily supported by artificial intelligencealgorithms, better simulating the propagation channels of the macro economyinto the financial institutions business models. Our vision will be to provide astress testing framework that succeeds to perform as an early warning systemfor financial shocks on individual banks’ balance sheets.This study is organized as follows. In section 2, we focus on the relatedliterature review on financial institutions stress testing. Section 3 describes thedata collection and processing. In section 4 we provide details regarding theestimation process of the various stress testing frameworks examined in thisstudy. In section 5 we compare across methodologies and provide experimentalresults using a test dataset of financial balance sheet sequences of data. Thisway, we assess not only the applicability of the proposed approach but also itsgeneralization capacity. Finally, in the concluding section 6 we summarize theperformance superiority of the proposed methodology, we identify any potentialweaknesses and limitations, while we also discuss areas for future researchextensions.A typical stress testing engine is composed by four elements: the perimeterof risks subjected to stress, the scenario design, the calculation engine thattransforms the shocks into an outcome in Banks balance sheet, and a measureof the outcome [5]. Reviewing current stress testing frameworks performedby regulatory authorities they follow broadly this structure. In particular themost famous stress testing exercises currently publicly available are: EBA [6],4 olvency Risk Solvency RiskLiquidity Risk Network SpilloverEffects Aggregate LossesStressScenarioStart of year ( t ) Interim date:6 months ( t ) End of year ( t )CET CET Figure 1: Feedforward architecture of currently established stress testing frameworks.
CCAR (FED) [7], PRA - Bank of England [8], ECB (top down) [9, 10], Bank ofCanada [11], Central Bank of Austria (ARNIE) [12], IMF [13], Bank of Greece(Diagnostic Exercise) [14]. The structure of all these exercises follows a left toright flow to estimate the impact of an adverse shock in the economy. One of thebasic components of all these exercises is the time horizon they estimate futurelosses for the banks which ranges between 2-5 years. During this period themacro economic scenario is given. This set of macro scenarios is passed throughto financial institutions to project their P&L and Risk Weighted Assets (RWA)and eventually estimate capital using regulatory hurdle rates. Some of theseexercises include in their structure a second round effects mechanism for thebanking system to account for contagion risk. Macroeconomic feedback effectsi.e for example the impact of a significant institution becoming insolvent in themacro economy, usually they are not considered in these frameworks. Stresstests under this structure can mainly serve as a tool to challenge the recoveryplans of banks and to assess their viability. But their role as an early warningsystem is questionable.As Drehmann [15] aptly points, systemic banking crises are reflected in theperformance of credit and property prices and usually they appear at the high5oint of the medium-term financial cycle. Therefore, crisis starts before itsdepicted in macro scenarios. According to Borio [5] a system is not fragile whena large financial shock materializes but when even a small negative change infinancial and macro variables is amplified through the different dynamic systemrelationships and can lead to a systemic shock. For example after the defaultof Lehman the financial market crashed and the US GDP exhibited a sharpdecrease causing a structural break in the macro data time series.Current versions of stress tests possess a macro scenario over time in a staticway without modelling or tracking in a path dependent nature the multistepdecision process and financial behaviour that in reality takes place from all eco-nomic participants. [16] Furthermore, non-linearity is not modelled adequately inthe statistical techniques currently employed. Risks under the current globalizedmarket tend to be amplified when a stress event occurs. Non-linear relationshipkicks in through channels of amplification leading to a chain of events unpre-dictable from the static nature of stress test. Under stressed conditions therelationships between modelled variables are non-linear [17] [18] and exhibitstructural breaks [19]. Stress testing frameworks are composed by standalonemodels usually combined in a qualitatively manner. A small single-step predic-tion error at the beginning could accumulate and propagate when combinedwithout taking the correlation of the financial variables, often resulting in poorprediction accuracy. Furthermore standalone models can lead to double countingeffects or overestimating the impact stemming from the changes of the predefinedmacro variables. Finally univariate setups are not able to model adequatelycomplex distributed variables with non-linear behaviour.In addition regular stress testing frameworks exhibit simplification assump-tions that may affect the reliability of the final estimation. EBA EU wide stresstesting is a bottom up exercise covering only specific risk on banks individualsbalance sheet based on a macro scenario usually based on simplified assump-tions. One of the weaknesses in EBA methodology is the static balance sheetassumption i.e assets and liabilities remain constant over the horizon withoutacknowledging for management actions and new generation of loans. In addition6itigation actions are taken into account after the stress testing are finalizedthrough a strong qualitative overlay and not in a dynamic way. [6]On the other hand System wide stress testing exercises on micro prudentiallevel are heavily relied upon on the interaction with individual banks with respectto data analytics and propagating the macro scenarios to their balance sheet.Thus estimations are not performed in a uniform statistical process but inheritthe model deficiencies and forecasts errors embedded in banks individual models.The heterogeneity in the results increases significantly the estimation errors andthere is no robust process for regulators to account for it. Thus the need forindependent central modelling for simulating the financial system is of greatconcern [20]. Furthermore the stress testing process involves the disclosure ofthe methodological framework to all market participants which in turn there areevidence of second round effects regarding the accounting treatment of banks.Specifically based on a study [21] banks participating in regulatory exercisestend to manipulate their provisions for credit risk to absorb the impact of theupcoming stress test.Finally Stress Testing outcomes in the current regulatory exercises heavilyrely on regulatory ratios like capital adequacy ratio which in turn is highlydependable on the estimation of RWA. Evidence in the literature [22] indicatesthat relaying in the risk weights applied internally by the financial institutionsunder the Basel Framework can lead to underestimation in capital needs. This isdriven by the significant variability stemming from internal models of the bankswhen applying internal model methods. Furthermore the regulatory frameworkcurrently employed for assessing the RWA cannot capture the hidden risk inbanks complex portfolio structure. In the current literature [23] there evidenceregarding especially more sophisticated banks (A-IRB) that they may performregulatory arbitrage and manipulates their true risks to lower their capitalrequirements. Thus robust macro modelling of the RWA using an independenttop down model is important to account for these cases.Although a significance progress in designing stress testing has been realizedin recent years, even today there are concerns that this type of exercises cannot7e used as early warning systems for financial distress [1]. By analysing thepublications regarding stress testing exercises either performed by regulators orindividuals banks we outlined a series of weaknesses and inefficiencies to provide aclear and concise view on the nature and on the way how the proposed approachin this study, DeepStress, attempts to address part of the aforementionedweaknesses.Deep neural networks architecture is one of the main innovations in ourproposed approach for dynamic balance sheet stress testing. Our approach isputting all the components together in a multivariate structure. We identifythe main channels of risk propagation in a recurrent form to account of all theexisting evidence of feedback effects in a financial institutions balance sheet. Thecurrent architectures is constrained by the use classical econometric techniqueswhich offer limited capabilities for simulating complex systems. Our approachaccounting for temporal patterns in banks balance sheets provides a dynamicmodelling approach. This is achieved through the multivariate training of deepneural networks taking account the dynamic nature of banks metrics and thewhole structure of the banks balance sheet. The approach proposed is composedby multivariate input and output layers able to capture the cross correlationbetween balance sheet items and the macro economy. Training is performedas one big complex network minimizing estimation errors and double countingeffects among various financial variables.To account for non-linear relationships that materialize under adverse macroe-conomic conditions machine learning techniques like deep learning can providemore efficient estimations. Deep Neural networks based on academic literatureare capable of simulating real life phenomena where relationships are complex.Therefore, our proposed framework using multilayer deep networks envisagesin capturing the dynamics inherent in a financial distress. In addition the ar-chitecture of aims to capture the amplifications channels leading to structuralbreaks.The methodology applied relies only on publicly available data and modelsare developed in a uniform way thus making the process of validation and8rror correction more feasible to be performed centrally. In addition offers theopportunity to experiment on advanced statistical machine learning techniquesa need recognized also in the academic literature [24]. To sum up, our modellingapproach is balanced between capturing the determinants that strongly affectthe health of a financial institution, while at the same time developing a dynamicbalance sheet simulator engine for establishing an early warning system topredict bank failures under an adverse scenario. The modelling framework thatwe implement captures temporal dependencies in a banks financial indicatorsand the macro economy. At the same time, it explores up to 3 years of laggedobservations, which are assumed to carry all the necessary information to describeand predict the financial soundness of a bank, and combines their evolution withthe relevant macroeconomic indicators.
2. Data Collection and Processing
The dataset supporting this is study refers to the United States bankingsystem. Specifically, we have collected information on non-failed, failed andassisted entities from the database of the Federal Deposit Insurance Corporation(FDIC), an independent agency created by the US Congress in order to maintainthe stability and the public confidence in the financial system. The collectedinformation is related to all US banks, while the adopted definition of a defaultevent in this dataset includes all bank failures and assistance transactions ofall FDIC-insured institutions. Under the proposed framework, each entity iscategorized either as solvent or as insolvent based on the indicators provided byFDIC. Observations referring the failed banks are excluded from the analysissince stress testing is performed on healthy financial entities.The dataset covers the 2007-2015 period; a 9 years period with quarterlyinformation resulting in dataset with more than 175,000 records. The selectedtime period, seems to approximate a full economic cycle, in terms of the DefaultRate evolution. Fig. 2, shows the number of records included in each observationquarter and the corresponding default rate. From a supervisory perspective, most9 igure 2: USA financial institutions in the sample. Historical overview for the period 2008-2014of the failed entities (source: FDIC) of the financial institutions in the sample apply the standardized approach formeasuring the Credit risk weights assets based on the United States adaptationof the Basel regulatory framework [25].The dataset was split into three parts (Fig. 2). An in-sample dataset(Full in sample) that is comprised of the data pertaining to the 80% of theexamined companies over the observation period 2008-2013 amounting to 101.641observations. For performing hyper parameter tuning of deep neural networkswe define an out-of-sample dataset (validation sample), including the rest 20%of the observations for the period 2008-2013 amounting to 25.252 observations.This is useful for deep learning models, in which the training sample is used totrain various candidate models with different architectures and specifications,while the validation set is used for selecting the best parameter setup and avoidoverfitting in the training dataset. This way the generalization capabilities inother datasets of the final selected model increases substantially.Finally performance evaluation is performed on an out-of-time dataset (testsample) that spans over the 2014-2015 observation period reaching 48.756 obser-vations. In all cases, the dependent (target) variable is the Capital AdequacyRatio (CAR) of each bank in the end of the one year forecast horizon. Tosummarize, we performed model fitting using exclusively the available trainingsample prescribed above. To perform model selection, we employed five-fold10ross-validation, using predictive accuracy as our model selection criterion (CARprediction error). Performance evaluation results are assessed on the availabletest sample, to allow for evaluating the generalization capacity of the developedmodels.In developing our model specifications, we examine an extended set of vari-ables that fully describe the financial status of each bank in the sample. Inaddition to the above-mentioned variables, we have also included in the datasetquarterly observations of the most commonly used macro-economic variables.Macro variables are the main input in the models developed since they areimportant for scenario analysis under a stress testing framework. The currentmodel setup includes contemporaneous macro variables along with 3 year lags.The intuition for this approach is to build models for scenario prediction whichis the main methodology for Stress Testing modelling. The macro variablesincluded in the development are: • GDP: Gross Domestic Product growth • EXPORT: US Total Exports growth • GOVCREDIT: Government Credit to GDP • DEBT: US public debt to GDP • GOVEXP: US government expenditure to GDP • INFLAT: US inflation • RRE: House Price Index growth • UNR: Unemployment Rate • YIELD10Y: 10Y US sovereign bonds yields • STOCKS: US Stock index S&P 500 returnsThe relevant stress financial variables for simulating the profitability and therisk weighted assets of each financial institution are:11
NLOAN: Net loans exposure • DEP: Total Deposits • DDEP: Total domestic deposits • ASSET: Average Total Assets • EASSET: Average Total Earning Assets • EQUITY: Average Total Equity • LOAN: Average total loans • CFD: Deposits Cost of funding • YEA: Yield on earning assets • NFIA: Noninterest income to average assets • RW: Risk Weight Density • LOSS LOAN: Loss allowance to loans • RWA: Total risk weighted assets • CAR%: Total risk based capital ratioModelling for the evolution of the balance sheet is performed on the growthrate of 4 key financial items: Deposits, Total Earning Assets, Total Loans andTotal Assets. In order to capture the idiosyncratic characteristics of each financialentity, 3 year lags are included in the training process for each financial variable.In the final model setup the use of multiple years financial and macroeconomicvariables allows for capturing internal trends of key items of a bank’ balancesheet and also the degree each entity is affected by the status of US economy.12 . Model Development
The success of the stress testing exercises performed in the past by regulatoryauthorities was put under scrutiny by all market participants and the researchcommunity. In order to investigate the capabilities of the proposed approachfor stress testing against broadly used frameworks we simulate two additionalmethods for balance sheet forecasting to benchmark its performance. Specificallywe developed a constant balance sheet approach following the framework adaptedby EBA to perform EU wide stress tests [6] and a dynamic balance sheet approachsupport by a group of satellite models to forecast individual financial variablesused by other regulatory authorities like ECB for macro prudential stress testing.In this section we provide an overview of the overall setup of the study andtechnical details of the three individual approaches employed.
The main component of a micro prudential solvency stress testing frameworkis the projection of a financial institution capital adequacy ratio or recently theCET 1 ratio (Core Equity Tier I ratio). In this study, we develop a Deep NeuralNetwork structure which receives as input the Macro variables and Balance sheetcomponents mentioned in Section 2 and provides as output the balance sheetand profitability structure of the bank on one year horizon as measured by 9 corevariables namely Net loans, Deposits, Assets, Earning Assets, Deposits, Costof funding, Yield on earning assets, Noninterest income to assets, Risk WeightDensity and Cost of Risk Loss allowance to loans).We focus on the forecasting of the CAR ratio since CET-1 ratio was introducedunder Basel III and is not available throughout our dataset. Specifically, our aimis to project the in a one-year-ahead the CAR ratio of each financial institutionin the sample. CAR ratio by definition is the ratio of a banks capital over the riskweighted assets in each time point t. In order to simulate the core mechanics of astress testing framework we simulate the evolution of the key financial variablesof a financial institutions balance sheet. The main setup is that we project one13 nput Layer:Financial Metrics Hidden Layer Hidden Layer Balance SheetProjection
Figure 3: Stress Test Deep Neural Network architecture. year ahead the evolution of the capital and the the risk weighted assets in orderto forecast the one year ahead CAR. The approach followed to adjust the capitalin time t is given by the formula:Capital t =Earnings from Assets t − loans loss provisions t + Net fees and commisions t − cost of funding from deposits t + Capital t − In order to adjust the capital of each entity we model 8 key financial variables.The first four refer to the dynamic evolution of the balance sheet i.e the growthof the asset and liability side: the growth rate of Deposits, Total loans, TotalAssets, Total Earning Assets. The remaining 4 refer to the yield in the next yearof each item from the asset or liability side: cost of risk of loans, yield on earningassets, yield on deposits and yield of net fees and commissions of total assets.The RWA are adjusted in 3 different ways depending on the ST methodology.Specifically for deep learning we project the growth of the RWA, for satellitemodelling a dedicated model is trained to project the RW density of eachfinancial institution in the sample, while for the constant balance sheet approachwe assume RWA remain constant for one year.14 .2. Deep Learning
We implement a Deep Neural Network (henceforth DNN) to address theissue of dynamic balance sheet forecasting. Deep learning has been an activefield of research in the recent years, as it has achieved significant breakthroughsin the fields of computer vision and language understanding. In particular theyhave been extremely successful in diverse time-series modelling tasks as machinetranslation [26, 27] machine summarization and recommendation engines [28].However, their application in the field of finance is rather limited. Specifically, ourpaper constitutes one of the first works presented in the literature that considersapplication of deep learning to address the challenging financial modelling task offinancial balance sheet stress testing. Deep Neural Networks differ from ShallowNeural Networks (one layer) on the multiple internal layers employed betweenthe input values and the predicted result (Fig. 3).Constructing a DNN without nonlinear activation functions is impossible,as without these the deep architecture collapses to an equivalent shallow one.Typical choices are logistic sigmoid, hyperbolic tangent and rectified linear unit(ReLU). The logistic sigmoid and hyperbolic tangent activation functions areclosely related; both belong to the sigmoid family. A disadvantage of the sigmoidactivation function is that it must be kept small due to their tendency to saturatewith large positive or negative values. To alleviate this problem, practitionershave derived piecewise linear units like the popular ReLU, which are now thestandard choice in deep learning research ReLU. The activation layers increasethe ability and flexibility of a DNN to capture non-linear relationships in thetraining dataset. On a different perspective, since DNNs comprise a huge numberof trainable parameters, it is key that appropriate techniques be employed toprevent them from overfitting.Indeed, it is now widely understood that one of the main reasons behindthe explosive success and popularity of DNNs consists in the availability ofsimple, effective, and efficient regularization techniques, developed in the last fewyears. Dropout has been the first and the most popular regularization techniquefor DNNs [29]. In essence, it consists in randomly dropping different units of15he network on each iteration of the training algorithm. This way, only theparameters related to a subset of the network units are trained during eachiteration. This ameliorates the associated network overfitting tendency, and itdoes so in a way that ensures that all network parameters are effectively trained.Inspired from these merits, we employ Dropout DNNs with ReLU activations totrain and deploy feed forward deep neural networks. More precisely we employthe Apache MXNET toolbox of R1. We postulated deep networks that are upto five hidden layers deep and comprise various numbers of neurons. Modelselection using cross-validation was performed by maximizing the RMSE metricon the projected CAR.In our setup multivariate deep learning networks will learn the balance sheetof financial institutions separately generating yearly forecasts by the interactionsof layered neurons after receiving historical values of banks previous economicstates. This hierarchical transmission of observed data between cascadinglayers of abstraction can decompose the structure of a bank balance sheet andfoster the multivariate representation of the financial variables for capturing thecorrelations between various assets and liabilities. This provides the functionalityof simultaneously modelling the balance sheet as a whole instead of usingsatellite models of regular stress testing frameworks. This is feasible basedon the fact that DNN are composed of multiple features for input and outputcomplex representations. Deep learning can facilitate the dynamic balance sheetprojection approach through the non-linear relationships representations of eachlayer offering a more realistic approach for stress testing. Information flowsthrough the system as a vector of macro and financial variables describing thestate of both the bank and the macro economy at any time stamp during theforecast period. Specifically the input vector contains around 60 variables andthe output vector is composed of 9 variables. The DNN architecture employedis capable to model the lead lag relationships between macro variables banksvariables financial variables and sovereign variables. Finally in the DeepStressengine using the aforementioned multivariate forecasting setup on individualbalance sheet we model simultaneously the RWA evolution of the bank and16onnect it to the macro environment.
Conventional architectures compute point estimates of the unknown values(e.g., layers’ weights) without taking into consideration any prior informationand without any uncertainty estimation of the produced values. The Bayesianframework offers a flexible and mathematically solid approach to incorporate priorinformation and uncertainty estimation by explicitly employing model averaging.The Bayesian treatment of particular model has been shown to increase itscapacity and potential, while offering a natural way to assess the uncertainty ofthe resulting estimates. To this end, we augment the conventional architecturesof the previous sections, by relying on the Bayesian framework. Specifically, weimpose a prior Normal distribution over network weights, seeking to infer theirposterior distribution given the data. Since the marginal likelihood is intractablefor the considered architectures, from the existing Bayesian methods, we rely onapproximate inference and specifically on Variational Inference.Since the true posterior of the model cannot be computed, in VariationalInference, we introduce an auxiliary variational posterior distribution of a familyof distributions; we then try and find the optimal member of the consideredfamily to match the true posterior. This matching is achieved through theminimization of the Kullback-Leibler divergence between the true and theintroduced variational posterior. The KL divergence is a metric of similaritybetween two distributions and is a non-negative value; KL is zero, if and only if,the two considered distributions match exactly. Minimizing the KL divergenceis equal to the maximization of the Evidence Lower Bound, a well known boundon the marginal likelihood derived using Jensen’s inequality. Thus for trainingthe following architectures, we resort to ELBO maximization.
Local-Winner-Takes-All Mechanism.
The commonly employed nonlinearitiessuch as ReLUs are a mathematically convenient tool for training deep networksbut nevertheless do not come with a biologic plausibility. Research has shown17hat in the mammal brain, neurons with similar functionality and structure tendto group and compete with each other for their output. To this end, researchershave devoted significant effort to explore this type of competition betweenneuron and apply it in existing models. The resulting procedure is referred toas Local Winner-Takes-All and has been shown to provide competitive, or evenbetter results in benchmarks architectures in the image recognition domain [30].Thus, apart from the conventional ReLU activations of the previous section, weadditionally explore the potency of the LWTA to our domain. The linear unitsafter the affine transformation in each layer are grouped together and competefrom their outputs. This competition is performed in a probabilistic way, byemploying a softmax nonlinearity, obtaining the probability of activation of eachunit in each block.Specifically, let us consider input data X ∈ R N × D , containing N observations,with D features each. In a traditional hidden layer, we compute an inner productbetween the input and a weight matrix W ∈ R D × K , and the resulting activationis then passed through a non-linear function; the output of each layer is denotedas Y = { y n } Nn =1 , with y n ∈ R K . Thus, the corresponding procedure can bedescribed by the following computation: y nk = σ (cid:32) d (cid:88) d =1 w dk x nd (cid:33) (1)The most widely used non-linearity σ ( · ) is the Rectified Linear Unit (ReLU),such that for an input x : relu ( x ) = max(0 , x ) (2)In the LWTA approach, this mechanism is replaced by introducing blocks ofunits in each layer; therein, each unit is competing with the rest for its activation.The winner unit gets to pass its output to the next layer, while the rest arezeroed out. Now, the layer input is presented to each block; the weights of thelayer are reorganized in a three dimensional matrix, such that, W ∈ R D × B × U ,where B denotes the number of blocks and U the units in each block. Assuming18n input x ∈ R and weights W ∈ R D × U the output of each block and each unittherein, reads: y ku = h ku , if h ku ≥ h ki , ∀ i (cid:54) = u , otherwise (3)where h ku = σ ( w ku x ). A graphical illustration of the adopted approach ispresented in Fig. 4. We use the same architectures as in the previous experiments x x D ... ... ... ... LWTA layer
K K
LWTA layerInput layer Output layer
Figure 4: The LWTA architecture: Each layer comprises blocks of competing units; thereineach unit computes its activation and competes with the rest for its output. The winner get topass its output to the next layer, while the rest are zeroed out. for comparability and in order to assess any potential gains from the use of abiologically inspired mechanism.
Satellite models are used for univariate estimations of the impacts of stan-dalone balance sheet items in current stress testing frameworks [9]. A usualstatistical technique employed from regulators and by the banking industry isthe Bayesian Model Averaging. The main intuition behind the use of BMAeconometric technique is to account for the uncertainty surrounding the maindeterminants of risk dynamics especially in a period of recession. This approachis able to handle a short time series of balance sheet realizations which is usuallythe case for stress testing. Thus BMA offers the possibility to perform multivari-19te modelling including all potential predictors with different weight while theoutput of each trained model remains univariate.Using BMA, a pool of equations is generated using a random selectionsubgroup of determinants. Subsequently a weight is assigned to each modelthat reflects their relative forecasting performance. Aggregating all equationsusing the corresponding weights produces a posterior model probability. Thenumber of equations estimated in the first step is large enough to capture allpossible combinations of a predetermined number of independent variables. ThusBayesian Model Averaging addresses model uncertainty and misspecification inselected explanatory variables in a simple linear regression problem.To further illustrate BMA, suppose a linear model structure, with Y t being thedependent variable, X t the explanatory variables, α constant, β the coefficients,and (cid:15) t a normal error term with variance σ . Y t = α γ + β γ X γ,t + (cid:15) t (4) (cid:15) t ∼ N (0 , σ I ) (5)A problem arises when there are many potential explanatory variables in amatrix X t which transforms the task of selecting the correct combination quiteburdensome. The direct approach to inference in a single linear model thatincludes all variables is inefficient or even infeasible with a limited number ofobservations. It can lead to overfitting, multicollinearity and increased manualre-estimations to account for non-significant determinants. BMA tackles theproblemy estimating models for all possible combinations of { X } and constructinga weighted average over all of them.Under the assumption that X contains K potential explanatory variables,BMA estimates 2 K combinations, and thus, 2 K models. Applying Bayes’ Theo-rem, model averaging is based on the posterior model probabilities: p ( M γ ∪ Y, X ) = p ( Y ∪ M γ , X ) p ( M γ ) p ( Y ∪ X )= p ( Y ∪ M γ , X ) p ( M γ ) (cid:80) K s =1 p ( Y ∪ M s , X ) p ( M s ) (6)20n Equation (6), p ( Y, X ) denotes the integrated likelihood which is constant overall models and is thus simply a multiplicative term. Therefore, the posteriormodel probability (PMP) is proportional to the integrated likelihood p ( Y ∪ M, X ) which reflects the probability of the data given the model M . Thus, thecorresponding weight assigned to each model is measured by using p ( M γ ∪ Y, X )in Eq. (6).In equation (6), p ( M ) denotes the prior belief of how probable model M is before analyzing the data. Furthermore, to estimate p ( Y, X ) integration isperformed across all models in the model space and to estimate the probability p ( Y ∪ M, X ) integration is performed given model M across all parameterspace. By performing renormalization of the product in equation (6), PMPscan be inferred and subsequently the model’s weighted posterior distribution forestimator β is given by p ( β ∪ Y, X ) = K (cid:88) γ =1 p ( β ∪ M γ , Y, X ) p ( M γ ∪ X, Y ) (7)The priors, posteriors and the marginal likelihood employed in the estimationare described analytically in Appendix Appendix A. For model development,the same train set used for DNN is employed. Before applying the BayesianAveraging algorithm we remove and linearly interpolate the outliers. In BayesianModel Averaging estimation we employ unit information prior (UIP), which setsg=N commonly for all models. We use also a birth/death MCMC algorithm(20000 draws) due to the large number of covariates included since using theentire model space would lead to a large number of iterations. We fix the numberof burn-in draws for the MCMC sampler to 10000. Finally the models prioremployed is the “random theta” prior by Ley and Steel [31], who suggest abinomial-beta hyper prior on the a priori inclusion probability. This has theadvantage that is less tight around prior expected model size (i.e. the averagenumber of included regressors) so it reflects prior uncertainty about model sizemore efficiently. For robustness purposes we varied the used prior employing theFernandez [32] propositions but the results were not substantially different. In21 able 1: Comparison of the predicted one year ahead CAR by ST approach for all banks andonly for Large financial institutions (more than 200 billion in assets).
All banks in the dataset Out of Sample CAR In Sample CARSatellite Modelling (BMS) 20.61 17.07Deep Learning (MXNET) 18.01 17.89Deep Learning (Bayesian ReLU) 18.80 17.83Deep Learning (Bayesian LWTA) .
23 18 . Constant Balance Sheet 20.03 17.49Actual 19.33 18.73
Large Banks ( >
200 bl) Out of Sample CAR In Sample CARSatellite Modelling (BMS) 15.07 11.04Deep Learning (MXNET) 12.7 11.12Deep Learning (Bayesian ReLU) 13 . . .
43 12 . Constant Balance Sheet 15.11 11.48Actual 13.75 14.16 order to develop all the satellite models for this approach we employ the utilitiesof BMS R package . After the training process 9 BMS models are developed: 4for the growth of balance sheet items, 4 models are forecasting the yields of avarious assets and liabilities and one model for forecasting the RW assets density. For the constant balance all balance sheet items are assumed constant alongwith the RWA metric for one year. Thus we combine the respective univariatesatellite models BMA to project yields of assets and liabilities while assume zerogrowth in the balance sheet in order to project the CAR ratio one year ahead.
4. Model validation - Experimental Evaluation
No thorough and consistent framework exists for validating the results ofa stress testing exercise since the adverse scenario used in their design nevermaterialize. Back testing methods is an important process to recognize modellinginefficiencies and fine tune the estimations taking into account specificities in thetime series data that were not capture in the initial calibration and developmentphase. Thus in order to improve the quality of stress testing rigorous validationprocedures of actual vs predicted financial variables are important. Furthermore,according to academic literature the success of the stress testing exercises afterthe financial crises maybe be circumstantial [20] since no robust methods areapplied to quantify their estimation error. https://cran.r-project.org/web/packages/BMS/index.html Q Q Q Q Q Q Q Q Out-of-Sample Capital (Average over Samples)
ActualBMS MXNETConstant Bayesian ReLUBayesian LWTA
Figure 5: Out of sample back testing results of the Capital of the three balance sheet approachescompared with the actual figures (Whole Sample).
Following a different venue in this study we perform a thorough validationprocedure in order to assess the robustness of our approach. In this sectionwe summarize the results of the 5 approaches. More precisely, we report theperformance results obtained from the experimental evaluation of our methods,in terms of in-sample fit (train dataset) and out-of-time performance (testsample). To sum up, after developing our Stress testing frameworks in theIn-sample datase spanning the years 2010 to 2013 (16 quarters), we assess itsperformance under the Out-of-time dataset in which the performance of eachmodel is evaluated during a future time period for evaluating their generalization Q
12 0 1 0 Q
22 0 1 0 Q
32 0 1 0 Q
42 0 1 1 Q
12 0 1 1 Q
22 0 1 1 Q
32 0 1 1 Q
42 0 1 2 Q
12 0 1 2 Q
22 0 1 2 Q
32 0 1 2 Q
42 0 1 3 Q
12 0 1 3 Q
22 0 1 3 Q
32 0 1 3 Q . . . . . . . . . In-Sample fitted CAR (All Banks)
ActualBMS MXNETConstant Bayesian ReLUBayesian LWTA
Figure 6: In sample back testing results of CAR ratio of the balance sheet approaches (WholeSample). Q Q Q Q Q Q Q Q Out-of-Sample Projected CAR (All Banks)
ActualBMS MXNETConstant Bayesian ReLUBayesian LWTA
Figure 7: Out of sample back testing results of CAR ratio of the three balance sheet approaches(Large Banks in the out of Sample) capacity. More precisely, we report performance results obtained by evaluatingour method over a two year (8 quarters) out of sample time-period spanningfrom 2014 2015. Validation is performed with respect the one year aheadforecast of the CAR ratio. Note that the last 2 two years of the dataset werenot used for model development. Prediction accuracy of the CAR ratio, asmeasured by the deviation between the forecast of each framework against theactual CAR ratio of each financial institution, is the main criterion to assess theefficacy of each method and to select the most robust one. In this section, wepresent a series of metrics that are broadly used for quantitatively estimatingthe forecasting accuracy on continuous outcomes. We evaluate the stress testingmethods with the usual forecast metrics of Root Mean Square Error (RMSE),Mean Absolute Error (MAE) and the Mean Absolute Percentage Error (MAPE).These metrics are used so as to derive a full-spectrum conclusion regarding therelative forecasting power of each framework.As we observe in Table 1, Deep Learning algorithms provide the best empiricalfit both in-sample and out-of sample terms. Deep learning offers a more efficientand holistic way to simulate the CAR ratio under a specific set of macro scenariosof key macroeconomic variables as the predicted average CAR is closer to theactual one compared to Satellite Modelling and Constant Balance sheet stresstesting assumptions. Table 2 summarizes the results of all aforementioned24amples with respect the CAR ratio and the prediction error validation metrics(RMSE, MAE, MAPE). Based on the figures reported in the test sample DeepLearning algorithms provide more accurate estimation of the CAR ratio exhibitinga significant decrease in the forecasting error. Another remark based on theexperimental results is that, by moving from simple neural networks to Bayesiannetworks, we are able to infer richer and subtler dynamics from the data, thusincreasing our capacity in modelling nonlinearities and cross-correlations amongbalance sheet P&L items. This is also evident from Figs. 5 and 7 where the outof sample performance of constant balance sheet and satellite modelling divergesignificantly from the actual evolution of Regulatory Capital (Fig. 5) and theCAR ratio (Fig. 8) in the dataset even though they provide adequate fit in thedevelopent sample (Fig. 6). Contrary average CAR ratios estimated based onDeep Learning methods depict in an appropriate manner the CAR dymanics inthe projection period.To further investigate the performance of Deep Stress approach we narrowdown the results on a subset of large financial institutions where performanceof a robust stress testing methodology is more important due to their size andsocial-economic impact. Big financial institutions are defined as entities withmore than 200 billion in assets for the purpose of this study. Tables 1 and 2 also Q Q Q Q Q Q Q Q Out-of-Sample Projected CAR (Banks >
200 billion assets)
ActualBMS MXNETConstant Bayesian ReLUBayesian LWTA
Figure 8: Out of sample back testing results of the Capital of the three balance sheet approachescompared with the actual figures (Whole Sample). able 2: Comparison of the predicted one year ahead CAR by ST approach for all banks andonly for Large financial institutions (more than 200billions in assets) All banks Out of Sample (2014Q1-2015Q4)
RMSE MAPE MAE
Satellite Modelling (BMS) 11.32 2.88 0.15Deep Learning (MXNET) 11.18 2.36 0.12Deep Learning (Bayesian ReLU) 15.36 2.12 0.10Deep Learning (Bayesian LWTA) .
75 1 .
77 0 . Constant Balance Sheet 11.15 2.85 0.15In Sample (2010Q1-2013Q4)Satellite Modelling (BMS) 13.46 2.58 0.16Deep Learning (MXNET) 13.49 2.55 0.15Deep Learning (Bayesian ReLU) 16.58 2.41 0.15Deep Learning (Bayesian LWTA) 18.70 2.16 0.14Constant Balance Sheet 17.25 2.56 0.15 Large Banks ( >
200 bl) Out of Sample (2014Q1-2015Q4)
RMSE MAPE MAE
Satellite Modelling (BMS) 3.21 2.31 0.17Deep Learning (MXNET) 2.28 1.97 0.15Deep Learning (Bayesian ReLU) . .
51 0 . Constant Balance Sheet 3.56 2.58 0.19In Sample (2010Q1-2013Q4)Satellite Modelling (BMS) 3.44 3.14 0.23Deep Learning (MXNET) 3.46 3.13 0.22Deep Learning (Bayesian ReLU) 3.07 2.78 0.20Deep Learning (Bayesian LWTA) 2.76 2.42 0.18Constant Balance Sheet 3.27 2.94 0.21 shows that both in terms of fit and terms of error metrics the superiority of deepneural network is confirmed with significant drops in the forecasting error in thetest sample. Another worth mentioning results is the fact that although satelliteunivariate modelling in the sample dataset was expected to provide a betterfitting against the DNN this is not the case. DNN is trained in a multivariatesetup attempting to model 9 variables at the same time and still exhibits a rathercomparable in sample error against the other two methods. The same patternalso holds in Fig. 7 where the projected CAR is graphed only for this categoryof large banks (more than 200 billion in assets).Summarizing the results across all metrics in the test sample, it is evidentthat Deep Learning Algorithms exhibits higher predicting power compared toall the considered benchmark approaches. Among the other two approaches itis evident that the constant balance assumption although easier to implementexhibits the highest error. Hence, it is crucial for supervisory authorities torethink current stress testing exercises that are based on the constant balancesheet assumption and move towards a dynamic balance sheet approach.
5. Conclusions and Future Work
In this study we propose a new approach to be utilized in regulatory stresstesting exercises called Deep Stress that utilizes the properties of deep learning.26he main novel contribution of this empirical research to the literature offorecasting economic and financial crisis events is that we explore this newstatistical technique to tackle the problem of dynamic balance stress testing.Deep learning is utilized to provide a holistic modelling approach for a bankskey financial items. We perform thorough testing and validation of the proposedtechnique. Experimental results provide strong evidence to further be exploredin the future by regulators and financial institutions in order to produce anew generation of stress testing. Deep stress is compared with two broadlyaccepted stress testing frameworks: constant balance sheet and satellite dynamicmodelling.Summarizing our experimental results, we have found that Deep NeuralNetworks consistently outperform the benchmark approaches, Our analysis pro-vided strong evidence of increased forecasting accuracy with respect to the CARratio and performance consistency, which implies a much stronger generalizationcapacity compared to alternative benchmark frameworks. Validation measuresRMSE, MAE and MAPE significantly decrease in the test sample using Deep-Stress providing better simulation of the CAR ratio. Hence, these findings renderour approach much more attractive to researchers and practitioners workingin real-world financial institutions. The main driver for this higher forecastingaccuracy is the potential to model the balance sheet intercorrelation of P&Litems providing better simulation of the banks one-year-ahead activities. Sum-marizing, Deep Stress offers a better dynamic balance sheet simulator which is amajor component in any stress testing framework by better capturing that smallmacro and financial changes that can be amplified exponentially under a crisisevent. The holistic and dynamic nature of our framework leads to significantdecrease in the forecasting error by modelling better the feedback loops and theinterdependence of various items of a financial institution balance sheet with themacro economy.The aforementioned cascading layers structure of deep learning algorithms willopen up new horizons for financial system simulation combining brain inspiredcomputation and statistical machine learning. Of course our initial endeavour is27oncentrated on the banking system which the backbone of the global economybut is scalable to other entities as well like large corporate, insurances and shadowbanking. The system can be used by policy makers to test various measuresand to monitor the system in a forward looking manner. Our aim is to provideinnovation in the way regulatory authorities monitor the system and increaseawareness for possible future financial shocks. By simulating the complex nexusof the current financial establishment governments can proactively take stepsto mitigate any forthcoming adverse events. DeepStress can finally be used tomeasure the social impact of a possible financial or systemic shock through theadjusted projections of various key macro variables like unemployment, wealth,credit expansion etc.An aspect this work has not considered concerns developing deep learningmodels that can be continuously retrained in a moving window (online learning)setup. Another possible way forward is the exploration of deep neural networksunder a broad datasets referring to multiple jurisdictions. Finally, it is evidentthat the postulated Deep Learning networks can effectively capture nonlinearitiesin the relationship between the input variable and the output variable. Although,the validation framework cannot fully capture the estimation error in a Stresstesting exercise due to the fact the dataset does not include crisis years. Theresults though provide evidence for the forecasting efficacy of DeepStress forseveral years simulating a baseline scenario of a Stress Testing exercise. Toenhance the validation framework of our approach we will intensify the datacollection process to gather information referring to several years before thefinancial crises in order to use DeepStress to simulate and predict failed entitiesthat took place during this period. The value of such novel developments remainsto be examined in our future research endeavours.28 ppendix A.
Appendix A.1. Prior Selection In BMA models
It is a popular choice to set a uniform prior probability for each model torepresent the lack of prior knowledge. It is often the case in BMA to assumeno prior knowledge for each model and assign a uniform prior probability i.e. p ( M γ ) ∝
1. Regarding the marginal likelihoods p ( M γ ∪ Y, X ) and the posteriordistributions p ( β ∪ M γ , Y, X ), the literature standard is to use a specific priorstructure called Zellner’s g prior in order to estimate posterior distributions in anefficient mathematical way. In this setup the prior knowledge for the coefficientsis assumed to be a normal distribution with pre-specified mean and variance.Specifically the parametric formulation is given by Eq. (A.1): β γ ∪ g N (cid:18) , σ (cid:18) g X (cid:48) γ X γ (cid:19)(cid:19) (A.1)According to Eq. (A.1), coefficients are assumed to have zero mean and avariance-covariance structure which is broadly in line with that of the data X γ .The hyper-parameter g denotes the prior level of confidence that the coefficientsare zero. The posterior distribution of the coefficients follows a t-distributionwith expected value g g ˆ β γ , where ˆ β γ is the standard OLS estimator for model γ . Thus, as g → ∞ the coefficient estimator approaches the OLS estimator.Similarly, the posterior variance of β γ is affected by the value of g :Cov( β γ ∪ Y, X, g, M γ ) = ( Y − Y (cid:48) ) (cid:48) ( Y − Y (cid:48) ) N − g g × (cid:18) − g g R γ (cid:19) (cid:0) X (cid:48) γ X γ (cid:1) (cid:48) (A.2)The posterior covariance is similar to that of the OLS estimator, times a factorthat includes g and R γ .(OLS R squared for model γ ). For BMA, this priorframework results in a marginal likelihood which includes a size penalty factoradjusting for model size k γ given by: p ( Y ∪ M γ , X, g ) ∝ ( Y − y (cid:48) Y ) (cid:48) ( Y − Y (cid:48) y ) − N − (1 + g ) − kγ × (cid:18) − g g (cid:19) − N − (A.3)29he “default” approach for hyper-parameter g is the “unit information prior”(UIP), which sets g = N for all models. Acknowledgment
This work was co-funded by the European Regional Development Fundand the Republic of Cyprus through the Research and Innovation Foundation(Project: POST-DOC/0718/196).
References [1] I. Otker-Robe, A. Podpiera, The social impact of financial crises, WorldBank Policy Research Working Paper.[2] S. P. Chatzis, Forecasting stock market crisis events using deep and statisticalmachine learning techniques.” expert systems with applications 112.[3] T. Fischer, C. Krauss, Deep learning with long short-term memory networksfor financial market predictions.” european, Journal of Operational Research270.2 654669 78.[4] M. Kraus, S. Feuerriegel, Decision support from financial disclosures withdeep neural networks and transfer learning.” decision support systems 104.[5] C. Borio, M. Drehmann, K. Tsatsaronis, Stress-testing macro stress testing:does it live up to expectations?, BIS Working Papers 369, Bank for Interna-tional Settlements (Jan. 2012).URL https://ideas.repec.org/p/bis/biswps/369.html [6] Eba stress testing methodology 2018, 48, .[7] The fed - comprehensive capital analysis and review, .308] B. dAngleterre, A framework for stress testing the uk banking system, in:Bank of England Discussion Paper, London, p. 51.[9] S. Dees, J. Henry, Stress-Test Analytics for Macroprudential Purposes:Introducing STAMP, 2017, pp. 13–30.[10] J. Henry, M. Zimmermann, M. Leber, M. Kolb, M. Grodzicki, A. Amzallag,A. Vouldis, G. Haaj, C. Pancaro, M. Gross, P. Baudino, Sydow, A macrostress testing framework for assessing systemic risks in the banking sector,Occasional Paper Series 152, European Central Bank (Oct. 2013).URL https://ideas.repec.org/p/ecb/ecbops/2013152.html [11] K. Anand, G. Bdard-Pag, V. Traclet, Stress testing the canadian bankingsystem: a system-wide approach, Financial System Review 61.[12] M. Feldkircher, Arnie in action: the 2013 fsap stress tests for the austrianbanking system, Financial stability report 26 100118.[13] M. M. Hasan, C. Schmieder, C. Puhr, Next generation balance sheet stresstesting.[14] Greek banking system diagnostic exercise 2013, .[15] M. Drehmann, C. Borio, K. Tsatsaronis, Characterising the financial cycle:dont lose sight of the medium term!, in: Reserve Bank of Chicago-ECB14th Annual International Banking Conference, The role of central banksin financial stability, how has it changed?, Chicago, 10-11 November. 57.[16] R. Bookstaber, Stress tests to promote financial stability: Assessing progressand looking to the future.” journal of risk management in financial institu-tions 7.1.[17] M. Drehmann, A. Patton, S. Sorensen, Non-linearities and stress testing”,in risk measurement and systemic risk, in: Proceedings of the fourth jointcentral bank research conference, ECB.3118] M. Juselius, M. Kim, Sustainable financial obligations and crisis cycles”,Helsinki Economic Centre of Research Discussion Papers 313 55.[19] R. Alfaro, M. Drehmann, Macro stress tests and crises: What can we learn?,BIS Quarterly Review 2941 56.[20] B. Hirtle, A. Lehnert, Supervisory stress tests. annual review of financialeconomics 7.[21] D. Gounopoulos, J. Hbelt, N. I. Papanikolaou, Bank stress tests: An activetreatment or a placebo?[22] Bis stats, http://stats.bis.org/bis-stats-tool/org.bis.stats.ui.StatsApplication/StatsApplication.html .[23] G. Ferri, V. Pesic, Bank regulatory arbitrage via risk weighted assetsdispersion.” journal of financial stability(2016.[24] H. Oura, L. B. Schumacher, Macrofinancial stress testing-principles andpractices.[25] A. Castro Carvalho, Proportionality in banking regulation: a cross-countrycomparison.” fsi insights on policy implementation 1.[26] K. Cho, B. Van Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares,H. Schwenk, B. Y, Learning phrase representations using rnn encoderdecoderfor statistical machine translation, in: Proc. EMNLP, p. 81.[27] Z. Tu, Z. Lu, L. Y., X. Liu, H. Li, Modeling coverage for neural machinetranslation, in: Proc. ACL, p. 82.[28] M. Quadrana, B. Hidasi, A. Karatzoglou, P. Cremonesi, Personalizingsession-based recommendations with hierarchical recurrent neural networks,in: Proc. ASM RecSys, p. 83.[29] N. Srivastava, J. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov,Dropout: A simple way to prevent neural networks from overfitting, Journalof Machine Learning Research 15 19291958 84.3230] K. Panousis, S. Chatzis, S. Theodoridis, Nonparametric Bayesian deepnetworks with local competition, Vol. 97 of Proceedings of Machine LearningResearch, PMLR, Long Beach, California, USA, 2019, pp. 4980–4988.URL http://proceedings.mlr.press/v97/panousis19a.htmlhttp://proceedings.mlr.press/v97/panousis19a.html