Tak-chung Fu

Stock time series pattern matching: Template-based vs. rule-based approaches

One of the major duties of financial analysts is technical analysis. It is necessary to locate the technical patterns in the stock price movement charts to analyze the market behavior. Indeed, there are two main problems: how to define those preferred patterns (technical patterns) for query and how to match the defined pattern templates in different resolutions. As we can see, defining the similarity between time series (or time series subsequences) is of fundamental importance. By identifying the perceptually important points (PIPs) directly from the time domain, time series and templates of different lengths can be compared. Three ways of distance measure, including Euclidean distance (PIP-ED), perpendicular distance (PIP-PD) and vertical distance (PIP-VD), for PIP identification are compared in this paper. After the PIP identification process, both template- and rule-based pattern-matching approaches are introduced. The proposed methods are distinctive in their intuitiveness, making them particularly user friendly to ordinary data analysts like stock market investors. As demonstrated by the experiments, the template- and the rule-based time series matching and subsequence searching approaches provide different directions to achieve the goal of pattern identification.

Representing financial time series based on data point importance

Recently, the increasing use of time series data has initiated various research and development attempts in the field of data and knowledge management. Time series data is characterized as large in data size, high dimensionality and update continuously. Moreover, the time series data is always considered as a whole instead of individual numerical fields. Indeed, a large set of time series data is from stock market. Stock time series has its own characteristics over other time series. Moreover, dimensionality reduction is an essential step before many time series analysis and mining tasks. For these reasons, research is prompted to augment existing technologies and build new representation to manage financial time series data. In this paper, financial time series is represented according to the importance of the data points. With the concept of data point importance, a tree data structure, which supports incremental updating, is proposed to represent the time series and an access method for retrieving the time series data point from the tree, which is according to their order of importance, is introduced. This technique is capable to present the time series in different levels of detail and facilitate multi-resolution dimensionality reduction of the time series data. In this paper, different data point importance evaluation methods, a new updating method and two dimensionality reduction approaches are proposed and evaluated by a series of experiments. Finally, the application of the proposed representation on mobile environment is demonstrated.

Evolutionary time series segmentation for stock data mining

Stock data in the form of multiple time series are difficult to process, analyze and mine. However, when they can be transformed into meaningful symbols like technical patterns, it becomes easier. Most recent work on time series queries concentrates only on how to identify a given pattern from a time series. Researchers do not consider the problem of identifying a suitable set of time points for segmenting the time series in accordance with a given set of pattern templates (e.g., a set of technical patterns for stock analysis). On the other hand, using fixed length segmentation is a primitive approach to this problem; hence, a dynamic approach (with high controllability) is preferred so that the time series can be segmented flexibly and effectively according to the needs of users and applications. In view of the fact that such a segmentation problem is an optimization problem and evolutionary computation is an appropriate tool to solve it, we propose an evolutionary time series segmentation algorithm. This approach allows a sizeable set of stock patterns to be generated for mining or query. In addition, defining the similarity between time series (or time series segments) is of fundamental importance in fitness computation. By identifying perceptually important points directly from the time domain, time series segments and templates of different lengths can be compared and intuitive pattern matching can be carried out in an effective and efficient manner. Encouraging experimental results are reported from tests that segment the time series of selected Hong Kong stocks.

Preventing meaningless stock time series pattern discovery by changing perceptually important point detection

Discovery of interesting or frequently appearing time series patterns is one of the important tasks in various time series data mining applications. However, recent research criticized that discovering subsequence patterns in time series using clustering approaches is meaningless. It is due to the presence of trivial matched subsequences in the formation of the time series subsequences using sliding window method. The objective of this paper is to propose a threshold-free approach to improve the method for segmenting long stock time series into subsequences using sliding window. The proposed approach filters the trivial matched subsequences by changing Perceptually Important Point (PIP) detection and reduced the dimension by PIP identification.

Adopting genetic algorithms for technical analysis and portfolio management

This research examines two different applications of the Genetic Algorithms (GA) in portfolio management. GA is adopted to determine the optimized parameters setting of different technical indicators and portfolio weighting. Besides the Traditional GA, the Hierarchical GA is also adopted in this research. Different algorithms and the usage of different numbers of technical indicators are evaluated in different economic situations. GA shows its optimization power over different tasks in portfolio management.

Discovering the Correlation between Stock Time Series and Financial News

It is always expected that a correlation exists between the movement of stock prices (technical analysis) and news sentiment (fundamental analysis). If we can determine such a correlation, further interesting research directions will certainly be generated. In this paper, a system prototype is proposed for investigating the correlation between stock prices and news sentiment. Our primary target market is Hong Kong and the system is customized for Chinese language. Different methods and the impacts of various design parameters are tested in the experiments.

Distributional Similarity Model for Multi-modality Clustering in Social Media

User generated content (UGC) has become the fastest growing sector of the WWW. Data mining from UGC presents challenges not typically found in text mining from documents. UGC can be semi-structured and its content can be very short and informal, containing relatively little content similar to a chat or an email conversation. In addition UGC can be viewed as a multi-modality data. These characteristics pose big challenges and research questions for scholars to cope with. To cluster UGC data, we can construct multiple contingency tables of modalities and employ the multi-way distributional clustering (MDC) algorithm. However, by considering a contingency table which summarizes the co-occurrence statistics of two modalities, it is not robust to represent the information entropy between two modalities in UGC data. In this paper, we propose a novel similarity measurement, called distributional similarity model (DSM), to solidify the graph model in the MDC algorithm to deal with the unique characteristics of the UGC data.

Analysis and Visualization of Time Series Data from Consumer-Generated Media and News Archives

Internet has become an indispensable part of everyday life with millions of people around the globe using it for a wide range of daily activities such as monitoring stock prices, posting blogs, and browsing online newspapers. Though a vast amount of information can be easily searched and obtained in seconds simply by pressing a click with a fingertip, the overflow of information popping up may not be something really relevant to what we need and therefore, it creates a headache to us when it comes to scanning and extracting relevant and useful information. Finding a wise way of extracting only the useful data for further analysis plays a significant role in promoting the efficient and effective use of the internet. In this paper, we present a system which performs the analysis and visualization of the emerging consumer generated media (CGM) posts and online news archives in a more user-friendly way. In order to overcome the heavy time complexity incurred, we would employ an approach to extract only the useful data from the CGM by means of the time series data processing technique, namely, the perceptual important point (PIP). By correlating the sorted out time series data with the online texts, further analysis could be done in a more effective and efficient way. With valuable and easy-to-understand information generated by using the perceptual important point (PIP), many businesses could gain the upper hand in todays competitive world market.

Adaptive data delivery framework for financial time series visualization

Nowadays, financial applications spread over various devices in both e-commerce and m-commerce. One of the major tasks in this kind of application is viewing the historical price movement of a stock for the market players before making any decision. However, due to the divergence of the configurations among different devices and platforms, adaptation is necessary for time series data delivery and visualization. In this paper, an adaptive framework of financial time series delivery and visualization is proposed to achieve this goal. The core system adapts the concept of data point importance and data points reordering for time series representation. The application of the proposed framework on stock price time series delivery and visualization is demonstrated in different devices using Web service.

Progressive time series visualization in a mobile environment

Beside their original functions, mobile devices are now enabled with increasing computational power and wireless communication technology that can facilitate many different applications. One valuable application is about the mobile access of financial data and a major task involved is to provide the historical stock price movement so that the market players can make appropriate investment decisions. However, due to the limitations of this new environment (e.g. screen size, bandwidth for data exchange, storage and computational power), adaptation is needed for time series visualization. In this paper, a progressive time series visualization method, with additional compression ability, for the mobile environment is proposed. The proposed method is based on a perceptually important point (PIP) identification scheme so that the time series data points can be disseminated in the order of importance for the changing (data access) requirements. The application of the proposed method to stock price time series visualization in mobile devices is demonstrated.