Haikun Hong

Deep Architecture for Traffic Flow Prediction: Deep Belief Networks With Multitask Learning

Traffic flow prediction is a fundamental problem in transportation modeling and management. Many existing approaches fail to provide favorable results due to being: 1) shallow in architecture; 2) hand engineered in features; and 3) separate in learning. In this paper we propose a deep architecture that consists of two parts, i.e., a deep belief network (DBN) at the bottom and a multitask regression layer at the top. A DBN is employed here for unsupervised feature learning. It can learn effective features for traffic flow prediction in an unsupervised fashion, which has been examined and found to be effective for many areas such as image and audio classification. To the best of our knowledge, this is the first paper that applies the deep learning approach to transportation research. To incorporate multitask learning (MTL) in our deep architecture, a multitask regression layer is used above the DBN for supervised prediction. We further investigate homogeneous MTL and heterogeneous MTL for traffic flow prediction. To take full advantage of weight sharing in our deep architecture, we propose a grouping method based on the weights in the top layer to make MTL more effective. Experiments on transportation data sets show good performance of our deep architecture. Abundant experiments show that our approach achieved close to 5% improvements over the state of the art. It is also presented that MTL can improve the generalization performance of shared tasks. These positive results demonstrate that deep learning and MTL are promising in transportation research.

Deep Architecture for Traffic Flow Prediction

Traffic flow prediction is a fundamental problem in transportation modeling and management. Many existing approaches fail at providing favorable results duo to 1shallow in architecture;2hand engineered in features. In this paper, we propose a deep architecture consists of two parts: a Deep Belief Network in the bottom and a regression layer on the top. The Deep Belief Network employed here is for unsupervised feature learning. It could learn effective features for traffic flow prediction in an unsupervised fashion which has been examined effective for many areas such as image and audio classification. To the best of our knowledge, this is the first work of applying deep learning approach to transportation research. Experiments on two types of transportation datasets show good performance of our deep architecture. Abundant experiments show that our approach could achieve results over state-of-the-art with near 3% improvements. Good results demonstrate that deep learning is promising in transportation research.

Deep process neural network for temporal deep learning

Process neural network is widely used in modeling temporal process inputs in neural networks. Traditional process neural network is usually limited in structure of single hidden layer due to the unfavorable training strategies of neural network with multiple hidden layers and complex temporal weights in process neural network. Deep learning has emerged as an effective pre-training method for neural network with multiple hidden layers. Though deep learning is usually limited in static inputs, it provided us a good solution for training neural network with multiple hidden layers. In this paper, we extended process neural network to deep process neural network. Two basic structures of deep process neural network are discussed. One is the accumulation first deep process neural network and the other is accumulation last deep process neural network. We could build any architecture of deep process neural network based on those two structures. Temporal process inputs are represented as sequences in this work for the purpose of unsupervised feature learning with less prior knowledge. Based on this, we proposed learning algorithms for two basic structures inspired by the numerical learning approach for process neural network and the auto-encoder in deep learning. Finally, extensive experiments demonstrated that deep process neural network is effective in tasks with temporal process inputs. Accuracy of deep process neural network is higher than traditional process neural network while time complexity is near in the task of traffic flow prediction in highway system.

Improving deep neural network ensembles using reconstruction error

Ensemble learning of neural network is a learning paradigm where ensembles of several neural networks show improved generalization capabilities that outperform those of single networks. For deep learning of multi-layer neural networks, ensemble learning is still applicable. In addition, characteristics of deep neural networks can provide potential opportunities to improve the performance of traditional neural network ensembles. In this paper, we propose an ensemble criterion of deep neural networks that is based on the reconstruction error and present two strategies to solve the most important issues in ensemble learning of neural networks: component dataset sampling and output averaging. Component training datasets are selected according to the reconstruction error instead of random bootstrap sampling or re-weighting. Moreover, for each testing instance, we can compute the reconstruction error yielded by the sub-model simultaneously with the output. The reconstruction error is used as the weights in output averaging. From the perspectives of prediction interval and confidence interval, we demonstrated that smaller reconstruction error could ensure smaller prediction interval. We also incorporate the famous structure ensemble approach “Dropout” into the proposed approach to achieve the best performance. We conduct experiments on classification and regression datasets to validate the effectiveness of our approach.

Metric-Based Multi-Task Grouping Neural Network for Traffic Flow Forecasting

Traffic flow forecasting is a fundamental problem in transportation modeling and management. Among various methods multi-task neural network has been demonstrated to be a promising and effective model for traffic flow forecasting, while there are still two issues unconsidered: 1) learning unrelated tasks together tends to reduce the model’s performance; 2) how to define or learn the distance metric for distinguishing related tasks and unrelated tasks. In this paper, a metric learning based K-means method is proposed to group related tasks together which effectively reduces the semantic gap between domain knowledge and handcrafted feature engineering. Then for each group of tasks, a deep neural network is built for traffic flow forecasting. Experimental results show the metric-based grouping method clusters tasks more reasonably with a better metric than classic Euclidean-based K-means. The final results of traffic flow forecasting on real dataset show the metric-based multi-task neural network outperforms the Euclidean-based multi-task neural network.

Learning Common Metrics for Homogenous Tasks in Traffic Flow Prediction

Nearest neighbor based nonparametric regression is a classic data-driven method for traffic flow prediction in intelligent transportation systems (ITS). Performances of those models depend heavily on the similarity or distance metric used to search nearest neighborhood. Metric learning algorithms have been developed to learn the distance metrics from data in recent years. In real-world transportation application, multiple forecasting tasks are set since there are lots of road sections and detector points in the traffic network. Previous works tend to learn only one global metric to be used for all the tasks or learn multiple local metrics for each task which may lead to under-fitting or over-fitting problem. To balance these two kinds of methods and improve the generalization of learned metrics, we propose a common metric learning algorithm under the intuition that homogenous tasks tend to have similar local metrics. Then the learned common metrics are used in common metric KNN (CM-KNN) for traffic flow prediction. Experimental results show that our algorithm to learn common metrics are reasonable and CM-KNN method for traffic flow prediction outperforms other competing methods.

Hybrid Multi-metric K-Nearest Neighbor Regression for Traffic Flow Prediction

Traffic flow prediction is a fundamental component in Intelligent Transportation Systems (ITS). Nearest neighbor based nonparametric regression method is a classic data-driven method for traffic flow prediction. Modern data collection technologies provide the opportunity to represent various features of the nonlinear complex system which also bring challenges to fuse the multiple sources of data. Firstly, the classic Euclidean distance metric based models for traffic flow prediction that treat each feature with equal weight is not effective in multi-source high-dimension feature space. Secondly, traditional handcrafting feature engineering by experts is tedious and error-prone. Thirdly, the traffic conditions in real-life situation are too complex to measure with only one distance metric. In this paper, we propose a hybrid multi-metric based k-nearest neighbor method (HMMKNN) for traffic flow prediction which can seize the intrinsic features in data and reduce the semantic gap between domain knowledge and handcrafted feature engineering. Experimental results demonstrate multi-source data fusion helps to improve the performance of traffic parameter prediction and HMMKNN outperforms the traditional Euclidean-based k-NN under various configurations. Furthermore, visualization of feature transformation clustering results implies the learned metrics are more reasonable.

Traffic Flow Decomposition and Prediction Based on Robust Principal Component Analysis

Research on traffic data analysis is becoming more available and important. One of the key challenges is how to accurately decompose the high-dimensional, noisy observation traffic flow matrix into sub-matrices that correspond to different classes of traffic flow which builds a foundation for traffic flow prediction, abnormal data detection and missing data imputation. While in traditional research, Principal Component Analysis (PCA) is usually used for traffic matrix analysis. However, the traffic matrix is usually corrupted by large volume anomalies, the resulting principal components will be significantly skewed from those in the anomaly-free case. In this paper, we introduce the Robust Principal Component Analysis (robust PCA) for decomposition. It can mine more accurate and robust underlining temporal and spatial characteristics of traffic flow with all kinds of fluctuations. We performed a comparative experimental analysis based on robust PCA with PCA-based method on a real-life dataset and got better decomposition performance. In the real-life dataset, results show that through robust PCA most of the large volume anomalies are short-lived and well isolated in the residual traffic matrix while PCA failed. In traffic flow prediction experiments based on decomposition, it shows that the result based on robust PCA outperforms the PCA and simple average. It provide adequate evidence that robust PCA is more appropriate for traffic flow matrix analysis. Robust PCA shows promising abilities in improving the accuracy and reliability of traffic flow analysis.

Mining Dependencies Considering Time Lag in Spatio-Temporal Traffic Data

Learning dependency structure is meaningful to characterize causal or statistical relationships. Traditional dependencies learning algorithms only use the same time stamp data of variables. However, in many real-world applications, such as traffic system and climate, time lag is a key feature of hidden temporal dependencies, and plays an essential role in interpreting the cause of discovered temporal dependencies. In this paper, we propose a method for mining dependencies by considering the time lag. The proposed approach is based on a decomposition of the coefficients into products of two-level hierarchical coefficients, where one represents feature-level and the other represents time-level. Specially, we capture the prior information of time lag in spatio-temporal traffic data. We construct a probabilistic formulation by applying some probabilistic priors to these hierarchical coefficients, and devise an expectation-maximization (EM) algorithm to learn the model parameters. We evaluate our model on both synthetic and real-world highway traffic datasets. Experimental results show the effectiveness of our method.

Short-term traffic flow forecasting: Multi-metric KNN with related station discovery

Nonparametric regression is a classic method for short-term traffic flow forecasting in Intelligent Transportation Systems (ITS). Feature space construction and distance metric selection are two important parts in nonparametric regression. Few of previous works have taken both these two aspects into account together. In addition, how to use information of related stations in network scale is a key to improve the performance of ITS. In this paper, we propose a novel three-stage framework based on KNN to handle the issues above for short-term traffic flow forecasting. In the first stage, the related origin stations and destination stations of target task are discovered from the whole traffic network. Then for each target task, a particular distance metric is learned in the second stage. Finally, an extended multi-metric k-nearest neighbor regression model is built in the third stage. Experimental results on real-world traffic dataset show that our multi-metric KNN model with Lasso outperforms the traditional KNN model and the feature construction method is effective.