Benjamin Van Vliet

Kaizen: Continuous Improvement

This chapter focuses on the concept of kaizen used in the development of trading systems. Kaizen in Japanese means continuous improvement. As a business strategy, it aims to eliminate waste in business processes. In the context of developing trading systems, kaizen is the process of continually searching for and implementing new quantitative methods, new data cleaning techniques, new optimization routines, new technologies, and new risk management methods that lengthens the maturity stage of a trading/investment system. Several methodologies exist for continuous improvement. The most widely known is the four-step quality model known as plan-do-check-act (PDCA) cycle. Other widely used methods, such as Six Sigma, Lean, and Total Quality Management, extend the PDCA model to emphasize management involvement and teamwork, measuring processes, and reducing waste and lowering cycle times. Six Sigma models for process improvement include DMAIC and DMADV. DMAIC stands for Define-Measure-Analyze-Improve-Control, which is applicable to an existing system. DMADV stands for Define-Measure-Analyze-Design-Verify, which is applicable to new systems. When applied to a trading environment, a continuous improvement strategy involves management, kaizen teams, and reformulated product teams, working together to make small improvements continuously. It is thus top managements responsibility to cultivate a professional environment that engenders kaizen as a culture of sustained continuous improvement focuses efforts on optimizing trading/investment systems.

13 – STAGE 2: Overview

This chapter presents an overview of the backtesting process, which is essentially the make or break stage of the trading/system design and development project. Performance of the system at the end of this stage will be the performance of the working system. Either the system generates acceptable performance or it does not. If the system does not, then the project is stopped or sent back to Stage 1. This spiral also contains the highest risk as testing migrates from a very small sample data set used to prove components in Stage 1 to a real set of data and all of the issues that go along with real data. Data can make or break a trading/investment system and selecting, cleaning, and filtering out bad data is the key stage in the development cycle, which will be important for risk management in Stage 4 as well. The initial test data could be an anomaly versus real data that does not prove the concept. Product teams often try to force the system to work with the real data, attempting to find anomalous data where it proves successful. After this loop, the building of the system is a straightforward process since only the interfaces and algorithm control (SPC and risk management) system are left to be completed.

Perform In-Sample/Out-of-Sample Tests

This chapter focuses on performing proper in-sample and out-of-sample tests, which is perhaps the most critical step in the trading/investment system development process. If the system is profitable both in sample and out of sample, it is likely to receive capital to begin implementation and trading as soon as possible. If a system is profitable only in sample, it may be allocated additional resources for continued research and/or sent back to Stage 1. If, however, the system proves to be unprofitable both in sample as well as out of sample, management will likely scrap the project altogether mainly due to the nonscalability of the trading idea. In-sample testing is very time intensive, as the team manually checks the calculations and results. During the in-sample test, algorithms may calculate the averages and standard deviations for trades. This is the step where the team converts all the prototype examples into prototype production-level code. In-sample testing also exposes irregularities in the data, so that the development team can make necessary modifications. Out-of-sample testing is done to ensure everything is working properly, with no adjustments and with almost real-world data and samples. During the out-of-sample testing, no more modifications can be made. Trading algorithms and quantitative models are examined against both in-sample and out-of-sample data before progressing to the implementation stage, so it is of utmost importance to save some of the historical data for out-of-sample testing.

Define Performance Controls

With the choice of so many methods to calculate performance metrics and the potential for errors and abuse, top management must establish internal controls and policies with respect to calculations, performance presentation standards, and independent verification in accordance with CFAI/GIPS/NAPF guidelines. Determining which metrics to calculate depends on the nature of the system. However, every system should be compared against a benchmark alternative. To make sure that a trading system consistently outperforms its benchmark, attribution analysis is performed on the portfolio and the benchmark. For attribution analysis, a risk manager must understand where the risk and return is versus the benchmark. The best-known approach to performance attribution is the Brinson–Fachler method, which uses weighted sums, compounding, and value added for performance analysis. The Brinson–Fachler model also looks at how the sector did in comparison with the benchmark and reports the results. The Monte Carlo simulation approximates the behavior of instrument prices by using computer generated random numbers to build hypothetical price paths through time. The VaR can be read directly from the distribution of simulated portfolio values and the main advantage of the Monte Carlo method is that it is able to cover a wide range of possible values in financial variables and fully account for correlations. This method is by far the most powerful method to calculate Value at Risk. It can consider any number of financial risks such as nonlinear price risk, volatility risk, model risk, credit risk, time varying volatility, non-normal distributions and fat tails, implied parameters, and extreme or user-defined scenarios.

STAGE 3: Overview

This chapter presents an overview of the Stage 3 of the development of trading and investment systems. This stage accomplishes the transformation task, turning the specifications of the trading/investment system into software and hardware, where the design of the trading system from previous stages becomes the specification requirements for the software design. At this stage, the task of building software and hardware components of the trading/investment system include a development team in addition to the product team. The lead programmer/IT member of the product team manages this somewhat agile development team, which will engage in test-driven development. The financial engineers from the product team over the course of Stage 3 manage software quality assurance of business tier software. The traders from the product team perform and/or oversee quality assurance over GUI, integration, and reporting mechanisms, as well as run user-acceptance tests. The marketing person from the product team writes or oversees the writing of the user documentation for the software. Addition of a risk manager, who will learn the specifics of the trading/investment system and be able to recommend how best to build out the risk management functions and reporting requirements, is also recommended. A development team develops software iteratively, and this iterative development dramatically increases feedback versus sequential development, providing closer communication between the product team and the development team.

25 – STAGE 4: Overview

This chapter presents an overview of risk management and portfolio management during the development of trading and investment system. The primary difference between the classical view of risk and the risk faced in the systematic trading/investment world is that while the former attempts to manage a person such as a trader or portfolio manager, the latter relates to manage the risk associated with a machine, which has predetermined performance and predetermined risk/reward ratios. In this stage, a product team is not involved anymore and in their place, risk managers and a kaizen team assumes control of the system. Risk managers will monitor the systems outputs and the kaizen team will make and/or implement containments, which are short-term solutions, not fixes or corrective actions, investigate root causes, and make recommendations for long-term fixes of the root cause. The machine has predetermined performance and predetermined risk/reward ratios, which are stochastic outputs that should be monitored by using SPC. The purpose of SPC is to notify risk managers of a problem in the machine, to make them aware of special, unassigned but assignable, variation in the system. The task of the risk managers and the product team, then, is to determine the root cause of the special variation in the performance of the machine. The demands on a risk manager are naturally more complex, as he has to be a part of a continuous improvement team, which is tasked by the management to fix trading/investment machines. Some of the deliverables used in this stage include return attribution and risk attribution report, root cause analysis reports, and identification of future risk distributions.

3 – Overview of the Trading/Investment System Development Methodology

This chapter presents an incremental development methodology of trading and investment system, which would ensure a rapid development cycle with quick deliverables and consistent quality standards. However, before the beginning of development methodology, the product team has to raise money to fund the development and to raise money, a product team needs a Money Document. Trading and money management is a business and in order to succeed, product teams need to raise research and development capital as well as trading capital, from either inside the firm or outside it. A well-done Money Document serves as a Vision and Scope Document by outlining the business goals of a proposed trading/investment system in a clear and concise fashion in order to persuade management or outside investors to provide the initial capital needed for research and subsequently for development of a trading/investment system according to the four-stage methodology. The resources allocated to a project, subsequent to the delivery of a Money Document and management approval, enables the process of development methodology. The methodology combines the various aspects of well-known methodologies such as the traditional waterfall of Royce and spiral methodology of Boehm from software, the Stage-Gate ® methodology of Cooper from new product development, and Lean, Six Sigma, and Agile development. The Royces traditional waterfall methodology for software development consists of four stages that include analysis, design, implementation, and testing that vaguely map to the four stages of the methodology. By using the waterfall methodology, the teams avoid the pitfalls of creating systems before project plans are precisely defined and approved. In the spiral methodology, a smaller amount of time is initially devoted to the four stages that include research, planning, implementation, and testing, which are followed by several iterations or loops over each.

Check Performance and Probationary Trade

This chapter focuses on the use of quality assurance (QA) for checking performance of the development system. QA of both the development process and the developed products ensure that development has been done properly and that the software and hardware components meet specifications and quality attribute requirements. The main objective of QA after each loop is to ensure that the product is ready to proceed with the next phase of development. Periodic audits, that is, at the end of each iteration, are a key technique used to operationalize QA. The product team should plan these audits in advance to be systematic. The first products audited should be the development teams iteration plans and documentation, and a comparison of the actual steps carried out with those in the documented plans. One purpose of an audit is to verify that the development teams status reports accurately reflect the status of development. QA assures that the development team performs formal software testing, such as unit, integration, and system testing, in accordance with plans and procedures. The documentation review includes test plans, test specifications, test procedures, and test reports and a proper QA monitors code reviews, inspections, and walkthroughs. Once the product team has signed off on all tests, probationary trading begins. Probationary trading finds out remaining design flaws in the trading/investment algorithm or the software prior to trading the full investment sum, which officially begins the track record period. The second purpose of probationary trading is to allow the traders time to use the GUI and reporting tools and determine the required additional tools to properly manage the trading/investment system. This is the final step in the testing stage of the trading system life cycle, and during this stage, when the product team checks performance, they use real-time and live trade data to measure.

Types of Trading Systems

This chapter focuses on the three broad categories of trading systems that include trigger systems, filter systems, and signal strength systems. Trigger trading systems are those where an opening position is taken upon the occurrence of a trigger, and where a closing trade is entered once another trigger occurs. In such a system, a technical indicator signals the opening and closing trades, which are dependent only on the instrument itself, and not on other instruments, positions, or market conditions. Technical systems can range from a very simple moving average crossover system to very complex high frequency trading systems. Filter systems select positions based upon a screen, or filter, and in such a system, an opening position may initially be taken based upon a trigger. However, unlike a trigger trade, a filter trade is dependent on other instruments, positions, or market conditions to set the filter parameters. The second step in a filter trade is portfolio management, which ranges from basic risk restrictions to attribution analysis. A signal strength, or multifactor, system ranks instruments based on factors or indicators such as P/E, P/S, or relative strength. Ranking stratifies the instruments into quartiles, deciles, or percentiles, and may be calculated relative to the investable universe or a subset of it. Ranked signals should be tested individually, independent of each other, using statistics to determine their predictive ability.

Prototype in Modeling Software

This chapter focuses on the role played by prototypes while modeling a trading and investment software. Prototyping either in Excel, Resolver, MATLAB, Zacks, Mathematica, SAS, and/or in other software allows the team to quickly prove complex mathematics and trading strategies in code. Converting mathematical models into prototype models leads to quick decisions and produces a defect-removal strategy with the greatest efficacy. They may vary in scope and complexity from being a small experiment to a final, consolidated version of the trading/investment system. The prototypes allow product teams to clarify calculations and expose inconsistencies in specifications, rapidly evaluate alternative methods, and deliver intermediate, working versions to end users and other interest parties for feedback. Excel, Resolver, and MATLAB are excellent tools for prototyping component algorithms and trading systems, but developers has to keep in mind that prototypes in Excel and MATLAB are programmed with an eye toward later conversion into object-oriented, programming code, like C++ or C#. So, Excel and VBA functions and algorithms should be fully documented and functions should be C-style and procedural in nature. While financial engineers and programmers will be involved in prototype development, the process must include and, in fact, should be overseen by someone trained in software testing. Enhancing the look of the spreadsheet with bold fonts, borders, shading, and graphs while displaying results is recommended as this will make the spreadsheet easier to read. While prototyping trading rules, one has to develop time series data so that all possible scenarios of changes in prices of the respective instruments can be investigated and modeled.