Xiangzhou Zhang

Software project risk analysis using Bayesian networks with causality constraints

Many risks are involved in software development and risk management has become one of the key activities in software development. Bayesian networks (BNs) have been explored as a tool for various risk management practices, including the risk management of software development projects. However, much of the present research on software risk analysis focuses on finding the correlation between risk factors and project outcome. Software project failures are often a result of insufficient and ineffective risk management. To obtain proper and effective risk control, risk planning should be performed based on risk causality which can provide more risk information for decision making. In this study, we propose a model using BNs with causality constraints (BNCC) for risk analysis of software development projects. Through unrestricted automatic causality learning from 302 collected software project data, we demonstrated that the proposed model can not only discover causalities in accordance with the expert knowledge but also perform better in prediction than other algorithms, such as logistic regression, C4.5, Naive Bayes, and general BNs. This research presents the first causal discovery framework for risk causality analysis of software projects and develops a model using BNCC for application in software project risk management.

Application of evolutionary computation for rule discovery in stock algorithmic trading

The first systematic literature review on evolutionary rule discovery in stock algorithmic trading.A clear demonstrate of studies in this field based on a classification framework.A precise analysis of gaps and limitations in existing studies based on detail of evaluation scheme.The most important factors influencing profitability of models are presented in detail.Targeted suggestions for future improvements based on the review are proposed. Despite the wide application of evolutionary computation (EC) techniques to rule discovery in stock algorithmic trading (AT), a comprehensive literature review on this topic is unavailable. Therefore, this paper aims to provide the first systematic literature review on the state-of-the-art application of EC techniques for rule discovery in stock AT. Out of 650 articles published before 2013 (inclusive), 51 relevant articles from 24 journals were confirmed. These papers were reviewed and grouped into three analytical method categories (fundamental analysis, technical analysis, and blending analysis) and three EC technique categories (evolutionary algorithm, swarm intelligence, and hybrid EC techniques). A significant bias toward the applications of genetic algorithm-based (GA) and genetic programming-based (GP) techniques in technical trading rule discovery is observed. Other EC techniques and fundamental analysis lack sufficient study. Furthermore, we summarize the information on the evaluation scheme of selected papers and particularly analyze the researches which compare their models with buy and hold strategy (B&H). We observe an interesting phenomenon where most of the existing techniques perform effectively in the downtrend and poorly in the uptrend, and considering the distribution of research in the classification framework, we suggest that this phenomenon can be attributed to the inclination of factor selections and problem in transaction cost selections. We also observe the significant influence of the transaction cost change on the margins of excess return. Other influenced factors are also presented in detail. The absence of ways for market trend prediction and the selection of transaction cost are two major limitations of the studies reviewed. In addition, the combination of trading rule discovery techniques and portfolio selection is a major research gap. Our review reveals the research focus and gaps in applying EC techniques for rule discovery in stock AT and suggests a roadmap for future research.

Stock trading rule discovery with an evolutionary trend following model

eTrend is a hybrid long-term and short-term evolutionary trend following model.eTrend generates high Sortino ratio and cumulative return after transaction cost.eTrend outperforms B&H strategy, as well as DT and ANN trading models.Concept drift analysis has identified adverse rules between bear and bull markets. Evolutionary learning is one of the most popular techniques for designing quantitative investment (QI) products. Trend following (TF) strategies, owing to their briefness and efficiency, are widely accepted by investors. Surprisingly, to the best of our knowledge, no related research has investigated TF investment strategies within an evolutionary learning model. This paper proposes a hybrid long-term and short-term evolutionary trend following algorithm (eTrend) that combines TF investment strategies with the eXtended Classifier Systems (XCS). The proposed eTrend algorithm has two advantages: (1) the combination of stock investment strategies (i.e., TF) and evolutionary learning (i.e., XCS) can significantly improve computation effectiveness and model practicability, and (2) XCS can automatically adapt to market directions and uncover reasonable and understandable trading rules for further analysis, which can help avoid the irrational trading behaviors of common investors. To evaluate eTrend, experiments are carried out using the daily trading data stream of three famous indexes in the Shanghai Stock Exchange. Experimental results indicate that eTrend outperforms the buy-and-hold strategy with high Sortino ratio after the transaction cost. Its performance is also superior to the decision tree and artificial neural network trading models. Furthermore, as the concept drift phenomenon is common in the stock market, an exploratory concept drift analysis is conducted on the trading rules discovered in bear and bull market phases. The analysis revealed interesting and rational results. In conclusion, this paper presents convincing evidence that the proposed hybrid trend following model can indeed generate effective trading guidance for investors.

A causal feature selection algorithm for stock prediction modeling

A key issue of quantitative investment (QI) product design is how to select representative features for stock prediction. However, existing stock prediction models adopt feature selection algorithms that rely on correlation analysis. This paper is the first to apply observational data-based causal analysis to stock prediction. Causalities represent direct influences between various stock features (important for stock analysis), while correlations cannot distinguish direct influences from indirect ones. This study proposes the causal feature selection (CFS) algorithm to select more representative features for better stock prediction modeling. CFS first identifies causalities between variables and then, based on the results, generates a feature subset. Based on 13-year data from the Shanghai Stock Exchanges, comparative experiments were conducted between CFS and three well-known feature selection algorithms, namely, principal component analysis (PCA), decision trees (DT; CART), and the least absolute shrinkage and selection operator (LASSO). CFS performs best in terms of accuracy and precision in most cases when combined with each of the seven baseline models, and identifies 18 important consistent features. In conclusion, CFS has considerable potential to improve the development of QI product.

An integrative framework for intelligent software project risk planning

Software projects have inherent uncertainties and risks. Social software projects suffer even more requirement changes and require more attention to risk management. Risk analysis and planning are complex, making it difficult to manage risks effectively through subjective judgment. At present, ample empirical research on intelligent decision-support models for risk analysis in software projects exists. However, to the best of our knowledge, empirical models for software project risk planning, or those related to integrative software risk analysis and planning are not available. Thus, the current study proposes an integrative framework for intelligent software project risk planning (IF-ISPRP) to help in minimizing the impacts of project risks and achieving a better foreseeable project outcome. IF-ISPRP includes two core components, namely, risk analysis module and risk planning module. The risk analysis module is to predict whether a project will be successful or not. The risk planning module is to produce a cost-minimal action set for risk control based on the risk analysis module. For integrative risk analysis and planning, we propose a novel many-to-many actionable knowledge discovery (MMAKD) method for complex risk planning. We also apply the framework on a social media platform project, Guangzhou Wireless City, and demonstrate how the model can generate a cost-minimal action set to mitigate the project risk. The risk-control actions found may help develop strategies on mitigating the risks of other social software projects. We hope that the proposed framework will provide an intelligent decision-support tool for project stakeholders to effectively control project risks by integrating risk analysis and planning.

An evolutionary trend reversion model for stock trading rule discovery

Quantitative investment (QI) is certainly a hot topic in big data analysis. For knowledge discovery in huge, complex and nonlinear stock market data, the eXtended Classifier Systems (XCS) is quite suitable because of the excellent learning and explicit expression abilities derived from its intrinsic techniques that include classification rule mining, evolutionary learning and reinforcement learning. This paper presents an Evolutionary Trend Reversion Model (eTrendRev), which is based on the proposed XCS with learn mode (XCSL) and trend-reversion strategy. The eTrendRev is highlighted in three aspects: (1) the explicit rules generated by XCSL are more understandable than black-box models, such as neural networks, thus can provide justifiable knowledge to guide trading; (2) the original pure explore mode of XCS is substituted by the proposed learn mode, which is shown in this study to perform better and is more stable; (3) a variety of trend-reversion strategies are integrated and made dynamic through evolutionary learning. For model evaluation, experiments were carried out on the historical data of the Shanghai Composite Index and the NASDAQ Composite Index, and back-testing results indicate that eTrendRev can produce higher return with lower risk and recognize significant market turning points in a timely fashion. This study also confirms the profitability of using sole trend-reversion indicators in machine learning-based QI model.

Cost-sensitive and ensemble-based prediction model for outsourced software project risk prediction

Nowadays software is mainly developed through outsourcing and it has become one of the most important business practice strategies for the software industry. However, outsourcing projects are often affiliated with high failure rate. Therefore to ensure success in outsourcing projects, past research has aimed to develop intelligent risk prediction models to evaluate the success rate and cost-effectiveness of software projects. In this study, we first summarized related work over the past 20years and observed that all existing prediction models assume equal misclassification costs, neglecting actual situations in the management of software projects. In fact, overlooking project failure is far more serious than the misclassification of a success-prone project as a failure. Moreover, ensemble learning, a technique well-recognized to improve prediction performance in other fields, has not yet been comprehensively studied in software project risk prediction. This study aims to close the research gaps by exploring cost-sensitive analysis and classifier ensemble methods. Comparative analysis with T-test on 60 different risk prediction models using 327 outsourced software project samples suggests that the ideal model is a homogeneous ensemble model of decision trees (DT) based on bagging. Interestingly, DT underperformed Support Vector Machine (SVM) in accuracy (i.e., assuming equal misclassification cost), but outperformed in cost-sensitive analysis under the proposed framework. In conclusion, this study proposes the first cost-sensitive and ensemble-based hybrid modeling framework (COSENS) for software project risk prediction. In addition, it establishes a new rigorous evaluation standard for assessing software risk prediction models by considering misclassification costs. Display Omitted The first cost-sensitive and ensemble framework to predict software project riskA comprehensive T-test method was used for rigorous performance comparison.A total of 60 models were built and compared based on 327 real project samples.Decision tree underperformed SVM in accuracy, but outperformed in cost analysis.A new rigorous model standard for software project risk analysis is established.

Multiple-cause discovery combined with structure learning for high-dimensional discrete data and application to stock prediction

Causal discovery in observational data is crucial to a variety of scientific and business research. Although many causal discovery algorithms have been proposed in recent decades, none of them is effective enough in dealing with high-dimensional discrete data. The main challenge is the complex interactions among large volume of variables, leading to numerous spurious causalities found. In this work, we propose a novel multiple-cause discovery method combined with structure learning (McDSL) to eliminate the spurious causalities. The method is carried out in two phases. In the first phase, conditional independence test is used to distinguish direct causal candidates from the indirect ones. In the second phase, causal direction of multi-cause structure is carefully determined with a hybrid causal discovery method. Validation experiments on synthetic data showed that McDSL is reliable in discovering multi-cause structures and eliminating indirect causes. We then applied this algorithm in discovering multiple causes of stock return based on 13-year historical financial data of the Shanghai Stock Exchanges of China, and established a stock prediction model. Experimental results showed that the McDSL discovered causes revealed changes of key risk factors of the stock market over 13 years, which indicated investors should change their investment strategy over time. Moreover, the causes discovered by McDSL have better performance in predicting stock return than that of other common filter-based feature selection algorithms.

iTrade: A Mobile Data-Driven Stock Trading System with Concept Drift Adaptation

Among all investors in the Chinese stock market, more than 95% are non-professional individual investors. These individual investors are in great need of mobile apps that can provide professional and handy trading analysis and decision support everywhere. However, financial data is challenging to analyze because of its large-scale, non-linear and noisy characteristics in a varying stock environment. This paper develops a Mobile Data-Driven Stock Trading System (iTrade), which is a mobile app system based on Client-Server architecture and various data mining techniques. The iTrade is characterized by 1) a data-driven intelligent learning model, which can provide further insight compared to empirical technical analysis, 2) a concept drift adaptation process, which facilitates the model adaptation to market structure changes, and 3) a rigorous benchmark analysis, including the Buy-and-Hold strategy and the strategies of three world-famous master investors (e.g., Warren E. Buffett). Technologies used in iTrade include the Least Absolute Shrinkage and Selection Operator (Lasso) algorithm, Support Vector Machine (SVM) and risk-adjusted portfolio optimization. An application case of iTrade is presented, which is based on a seven-year (2005-2011) back-testing. Evaluation results indicated that iTrade could gain much higher cumulative return compared to the benchmark (Shanghai Composite Index). To the best of our knowledge, this is the first study and mobile app system that emphasizes and investigates the concept drift phenomenon in stock market, as well as the performance comparison between data-driven intelligent model and strategies of master investors.

Concept drift mining of portfolio selection factors in stock market

Provide a model based on causal discovery technique (ANMCPT) for concept drift mining in cross-sectional analysis.AVMCPT can discover causal in high-dimensional and dynamic environment.ANMCPT outperform the classical Fama-French framework.Concept drift phenomenon in China stock market is observed and exhibited clearly. Concept drift is a common phenomenon in stock market that can cause the devaluation of the knowledge learned from cross-sectional analysis as the market changes over time in unforeseen ways. The widely used cross-sectional regression analysis based on expert knowledge has obvious limitations in handling problems that involve concept drift and high-dimensional data. To discover causal relations between portfolio selection factors and stock returns, and identify concept drifts of these relations, we apply a novel causal discovery technique called modified Additive Noise Model with Conditional Probability Table (ANMCPT). In evaluation experiments, we compares ANMCPT to the conventional cross-sectional analysis approach (i.e., Fama-French framework) in mining relationships between portfolio selection factors and stock returns. Results indicate that the factors selected by ANMCPT outperform the factors adopted in most previous cross-sectional researches that followed the Fama-French framework. To the best of our knowledge, this paper is the first to compare causal inference technique with Fama-French framework in concept drift mining of stock portfolio selection factors. Our causal inference-based concept drift mining method provides a new approach to accurate knowledge discovery in stock market.