Generative Adversarial Networks for Spatio-temporal Data: A Survey
Nan Gao, Hao Xue, Wei Shao, Sichen Zhao, Kyle Kai Qin, Arian Prabowo, Mohammad Saiedur Rahaman, Flora D. Salim
1111
Generative Adversarial Networks for Spatio-temporal Data: ASurvey
NAN GAO, HAO XUE, WEI SHAO, SICHEN ZHAO, KYLE KAI QIN, ARIAN PRABOWO,MOHAMMAD SAIEDUR RAHAMAN, and FLORA D. SALIM,
RMIT UniversityGenerative Adversarial Networks (GANs) have shown remarkable success in the computer vision area forproducing realistic-looking images. Recently, GAN-based techniques are shown to be promising for spatio-temporal-based applications such as trajectory prediction, events generation and time-series data imputation.While several reviews for GANs in computer vision been presented, nobody has considered addressingthe practical applications and challenges relevant to spatio-temporal data. In this paper, we conduct acomprehensive review of the recent developments of GANs in spatio-temporal data. we summarise thepopular GAN architectures in spatio-temporal data and common practices for evaluating the performanceof spatio-temporal applications with GANs. In the end, we point out the future directions with the hope ofbenefiting researchers interested in this area.CCS Concepts: •
Computing methodologies → Machine learning; Artificial intelligence;
Additional Key Words and Phrases: Generative adversarial nets, spatio-temporal data, time series, trajectorydata
ACM Reference format:
Nan Gao, Hao Xue, Wei Shao, Sichen Zhao, Kyle Kai Qin, Arian Prabowo, Mohammad Saiedur Rahaman,and Flora D. Salim. 2020. Generative Adversarial Networks for Spatio-temporal Data: A Survey.
ACM Comput.Surv.
37, 4, Article 111 (August 2020), 28 pages.DOI: 10.1145/1122445.1122456
Spatio-temporal properties are commonly observed in various fields, such as transportation [134],social science [75] and criminology [125], among which that have been rapidly transformed bythe proliferation of sensor and big data. The vast amount of spatio-temporal (ST) data requiresappropriate processing techniques for building effective applications. Generally, traditional methodsdealing with tabular data or graph data often perform poorly when applied to spatio-temporaldatasets. The reasons are mainly three-folds [145]: (1) ST data are usually in continuous spacewhile tabular or graph data are often discrete; (2) ST data usually present both spatial and temporalproperties where the data correlations are more complex to capture by traditional techniques; (3)ST data tends to be highly self-correlated and data samples are usually not independently generatedas in traditional data.With the prevalence of deep learning, many neural networks (e.g.,
Convolutional Neural Network (CNN) [74],
Recurrent Neural Network (RNN) [99],
Autoencoder (AE) [57],
Graph ConvolutionalNetwork (GCN) [69]) have been proposed and achieved remarkable success for modelling STdata. The wide adoption of deep learning for ST data is due to its demonstrated potential for
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without feeprovided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice andthe full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored.Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requiresprior specific permission and/or a fee. Request permissions from [email protected].© 2020 ACM. 0360-0300/2020/8-ART111 $15.00DOI: 10.1145/1122445.1122456 ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020. a r X i v : . [ c s . L G ] A ug hierarchical feature engineering ability. In this survey, we focus on one of the most interestingbreakthroughs in the deep learning field - Generative Adversarial Networks (GANs) [46] and theirpotential applications for ST data.GAN is a generative model which learns to produce realistic data adversarially. It consistsof two components [46]: the generator G and discriminator D . G captures the data distributionand produces realistic data from the latent variable z , and D estimates the probability of the datacoming from the real data space. GAN adopts the concept of the zero-sum non-cooperative gamewhere G and D are trained to play against each other until reaching a Nash equilibrium. GANshave gained considerable attention in various fields, involving images (e.g., image translation [62],super-resolution [76], joint image generation [87], object detection [32], change facial attributes[29]), videos (e.g., video generation [142]), natural language processing (e.g., text generation [86],text to image [159]).However, applying image or video generation directly are not applicable for modelling ST datasuch as traffic flow, regional rainfall, and pedestrian trajectory. On one hand, image generationusually takes the appearance between the input and output images into account, and fails toadequately handle spatial variations. On the other hand, video generation considers spatial dynamicsbetween images, however, temporal changes are not adequately considered when the prediction ofthe next image is highly dependent on the previous image [130]. Hence, new approaches need tobe explored for successfully applying GANs on ST data.Recently, GANs have started being applied to ST data. The applications for GANs on ST datamainly include the generation of de-identified spatio-temporal events [64, 130], time series imputa-tion [92, 93], trajectory prediction [53, 73], graph representation [15, 143], etc. Despite the successof GANs on computer vision area, applying GANs to ST data prediction is challenging [130]. Forinstance, leveraging additional information such as Places of Interest (PoI) , weather informationis still untouched in previous research. Besides, different to the images where researchers couldrely on visual inspections of the generated instances, evaluation of GANs on ST data remains anunsolved problem. It is neither practical nor appropriate to adopt the traditional evaluation metricsfor GAN on ST data [33, 130].A few research have reviewed recent literature on the problems in ST data or GAN applicationsin different fields. Compared with mining patterns from traditional relational data, modellingST data is particularly challenging due to its spatial and temporal attributes in addition to theactual measurements. Atluri et al. [10] have reviewed the popular problems and methods formodelling ST data. A taxonomy of the different types of ST data, ways of defining and describingdata instances has been provided to identify the relevant problems for any type of ST data inreal-world applications. They have also listed the commonly studied ST problems and reviewedthe issues for dealing with unique properties of different ST types. Want et al. [145] reviewed therecent progress in applying deep learning to ST data mining tasks and proposed a pipeline of theutilisation of deep learning models for ST data modelling problems. Hong et al. [60] explained theGANs from various perspectives and enumerate popular GAN variants applied to multiple tasks.Recent progress of GANs was discussed in [111] and Wang et al. [146] proposed a taxonomy ofGANs for computer vision area. Particularly, Yi et al. [153] reviewed recent advances of GANs inmedical imaging.However, all the above works reviewed either ST data modelling problems or the recent progressof GANs in the computer vision area. Though many researchers [33, 53, 92, 93, 130] have modelledST data with GANs, there is no related survey in this area to address the potential of using GANsfor ST data applications. For the first time, this paper presents a comprehensive overview of GANs
ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020. enerative Adversarial Networks for Spatio-temporal Data: A Survey 111:3 in ST data, describes promising applications of GANs, and identifies some remaining challengesneeded to be solved for enabling successful applications in different ST related tasks.To present a comprehensive overview of all the relevant research on GANs for ST data, we use
Google Scholar to conduct automated keyword-based search [123]. According to [6], GoogleScholar provides coverage and accessibility, and digital libraries such as IEEE Explore , ScienceDirect , ACM Digital Library . The search period is limited from 2014 to 2020 (inclusive) as theGAN has first appeared in 2014 [46]. However, papers that introduce novel concepts or approachesfor spatio-temporal data mining can be predated 2014. To ensure that our survey covers all relevantprimary literature, we have included such seminal papers regardless of their publication date.The remainder of the paper is organised as follows. In Section 2, we discuss the properties,characteristics and common research problems of ST data. We also present the popular deeplearning methods with non-GAN frameworks for ST data, including the Convolutional NeuralNetworks , Recurrent Neural Networks , Long Short-term Memory and
Gated Recurrent Units . Section3 reviews the definition of GANs and its popular variants with different architecture and lossfunctions. Section 4 lists the recent research progress for GANs in different categories of STapplications. Section 5 summarises the challenges on processing ST data with GANs, including theadapted architectures, loss functions and evaluation metrics. Finally, we conclude the paper anddiscuss future research directions.
The existence of time and space introduces a rich variety of spatio-temporal data types, leadingto different ways of formulating spatio-temporal data mining problems and techniques. In thispart, we will first introduce the general properties of spatio-temporal data, then briefly describethe common types of spatio-temporal data in different applications using generative adversarialnets techniques.
There are several general properties for spatio-temporal data (i.e., spatialreference, time reference, auto-correlation and heterogeneity [10]) described as below.
Spatial Reference . The spatial reference describes whether the objects are associated with thefixed location or dynamic locations [71]. Traditionally, when the data is collected from stationarysensors (e.g., weather stations), we consider the spatial dimension of the data is fixed. Recently,with the boost of mobile computing and location-based services, the dynamic locations of movingobjects have been recorded where the collected data comes from sensors attached to differentobjects, e.g., GPS trajectories from road vehicles [115].
Temporal Reference . The temporal reference describes to what extent the objects evolve [71].The simplest context includes objects do not evolve at all where only the static snapshots of objectsavailable. In a slightly more complicated situation, objects can change status but only the mostrecent update snapshot remains where the full history of status is unknown. The extreme contextconsists of moving objects where the full history of moving is kept, therefore generating time serieswhere all the status have been traversed.
Auto-correlation . The observations of spatio-temporal data are not independent and usuallyhave spatial and temporal correlations between near measurements. For example, in the trans-portation area, sensors in each parking lot with the unique spatial location can record the temporal https://scholar.google.com/ https://ieeexplore.ieee.org/ https://dl.acm.org/ ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020. (a) Spatio-temporal events AB (b) Two trajectories Fig. 1. Examples of spatio-temporal events and trajectories information when a vehicle arrives or leaves [116, 134]. This auto-correlation of spatio-temporaldata results in the smoothness of temporal measurements (e.g., temperature changes smoothlyover time ) and consistency between the spatial measurements (e.g., temperature values are similarin adjacent locations). Thereby, the traditional GAN techniques for computer vision field (e.g.,image generation [46]) without considering the temporal correlation may not well suited for thespatio-temporal data.
Heterogeneity . Spatio-temporal dataset can show heterogeneity in spatial or temporal infor-mation on different levels. For instance, traffic flow in a city can show similar patterns betweendifferent weeks. During a week, the traffic data on Monday may be different from data on Friday.There can also be inter-week changes due to public events or extreme weather, which can affectthe traffic patterns in a city. To deal with the heterogeneity of spatial and temporal information, itis necessary to learn different models for different spatio-temporal regions.
There are various spatio-temporal data types in real-world applications,differing in the representation of space and time context [10]. We describe the four common typesof spatio-temporal data which have been studied with GAN recently: (1) time series [18, 20, 33,55, 72, 79, 92, 93, 101, 166]; (2) spatio-temporal events [116, 130, 134]; (3) spatio-temporal graphs[77, 143, 152]; (4) trajectory data [53]. In this part, we provide a taxonomy of the data types availablein the spatio-temporal domain, then briefly discuss the properties of those data types and potentialdifficulties when facing with GANs.
Time Series . A time series can be represented as a sequence of data points X = { X , X , ..., X n } listed in an order of time (i.e., sequence of discrete-time data [140]). Examples of time series includethe values of indoor temperature during a day [38, 119], the changes of accelerometer readings inthe IoT devices [37, 39], fluctuations of the stock price in a month [166], etc. Time series analysisconsists of techniques to analyse time series for extracting useful statistic information and othercharacteristics of data. The common questions that used for dealing with time series include but notlimited to: Can we predict the future values for time series based on the historical values [72, 105, 147]?Can we cluster groups of time series with similar temporal and spatial patterns [5, 85]? Can we imputethe missing values automatically in multi-variate time series [93, 102]? Can we split time series intodifferent segments with its own characteristic properties [28, 63]?
Spatio-temporal Events . An spatio-temporal event represents a tuple containing temporal,spatial information as well as an additional observed value [82]. Generally, it is denoted as x i = { m i , t i , l i } , where t i and l i indicates the time and location of the event, m i means the value todescribe the event. Typically, the locations are recorded in three dimensions (i.e., latitude, longitude, ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020. enerative Adversarial Networks for Spatio-temporal Data: A Survey 111:5
Fig. 2. Example of Spatio-temporal Graph Data and altitude or depth), although sometimes only 1 or 2 spatial coordinates are available. Spatio-temporal events (see Fig. 1(a)) are frequently used in real-world applications such as the taxi demand[118], traffic flow [130], urban crimes [124], forest fires[27], etc. In some cases, spatio-temporalevents may even have duration like parking or heliophysics [113]. Usually, an ordered set ofspatio-temporal events can also be considered as an trajectory where the spatial locations visited bymoving objects. Some common questions that used for analysing spatio-temporal events includes:
Can we predict the future spatio-temporal events based on the previous observations [130]? Howare spatio-temporal events clustered based on time and space [135]? Can we identify the anomalousspatio-temporal events that do not follow the common patters of other events [12]?
Trajectory data . A trajectory represents the recordings of locations of a moving object atcertain times and it is usually defined as a function mapped from the temporal domain to thespatial domain [35]. Trajectories of moving points can be denoted as a sequence of tuples P = {( x , y , t ) , ( x , y , t ) , ..., ( x n , y n , t n )} , where ( x i , y i , t i ) indicates the location ( x i , y i ) at time t i .Several research have been conducted in the field of trajectory data mining and there are four majorcategories [163]: mobility of people [120], mobility of transportation [130], mobility of naturalphenomena and mobility of animals [83]. Fig. 1(b) shows an example of two trajectories of object A and object B . The common questions for processing trajectory data include: Can we predict thefuture trajectory based on the historical trajectory traces [53, 127, 128]? Can we divide a collectionof trajectories into small representative groups [133]? Can we detect the abnormal behaviours fromtrajectories [89]?
Spatio-temporal Graph . Spatio-temporal graph structure provides the representation of therelations between different nodes in different time. A sequence of spatio-temporal graphs [152]can be represented as G = (G , G , ..., G n ) where G i = { V i , E i , W i } indicates the graph snapshotat time T i ( i ∈ { , , ..., n } ). Spatio-temporal graphs have been applied in various domains suchas commerce (e.g., trades between countries [94]), transportation (e.g., route planning algorithms[42], traffic forecasting [155]) and social science (e.g., studying geo-spatial relations of differentsocial phenomena [51]). Fig. 2 is an example of spatio-temporal graphs in T , T , T . Some commonquestions for processing spatio-temporal graph includes: Can we forecast the status of graph basedon the historical graph representations [143, 155] ? Can we predict the links based on the previousgraph networks [77]?
This section introduces the traditional deep learning approaches for spatio-temporal data miningwith Non-GAN networks, including
Convolutional Neural Network , Recurrent Neural Network , Autoencoder , Graph Convolutional Network etc.
ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020.
Input Convolution+ReLu Pooling Convolution+ReLu Pooling Fully conneceted Output
Fig. 3. Basic structure of a typical CNN model ......
Unfold
Fig. 4. Basic structure of a typical RNN model
Convolutional Neural Network (CNN) [74] is a type of deep, feed-forward neuralnetwork commonly used to analyse visual imagery. Similar to other neural networks, a typicalCNN model is composed of an input layer, an output layer and some hidden layers as shown inFig. 3. Several commonly used hidden layers are convolution, Rectified Linear Unit (ReLU) [131]activation, pooling and fully connected layers. Convolutional layer put the previous layer througha series of convolutional filters and each filter activates certain features from the input. ReLU is anon-linear operation used after each convolutional layer, which replaces all negative values in thefeature maps by zero while maintaining positive pixel values. Pooling layer simplifies the outputfrom the previous rectified feature map through nonlinear down-sampling and parameter reduction.We take the maximal number of the input area when using the max-pooling layer and mean thenumber of the input area when using average pooling layer. After several convolutional, ReLU,and pooling layers, there are will be fully connected layers for high-level reasoning classification.Fully connected layers connect all neurons in the previous layer to every single neuron in the nextlayer, which is similar to the traditional multilayer perceptron (MLP). Compared to MLPs, CNNscan develop internal representations of two-dimensional images, which allows CNNs to be usedmore generally on other types of data with spatial correlations. Though CNNs are not specificallydeveloped for non-image data, it has been widely used in spatio-temporal data mining problem fortrajectory and spatio-temporal raster data [115].
Recurrent Neural Network (RNN) [99] is a type of neural networkswhere the previous outputs are fed as the input to the current step, which are widely used inNatural Language Processing (NLP) problems. The advantage of RNN is the hidden state (internalmemory) which captures information that has been calculated so far in a sequence. Fig. 4 showsthe basic architecture of a RNN, where X is the input data, y is the output data, h is the hiddenstate and U , V , W indicates the parameters of the RNN. The current state h t is calculated by thecurrent input X t and previous state h t − . ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020. enerative Adversarial Networks for Spatio-temporal Data: A Survey 111:7 tanh tanh
Fig. 5. Structure of a typical LSTM unit
Latent SpaceInput Data Output DataEncoder DecoderEncoded Data
Fig. 6. Structure of a typical Autoencoder
Though the RNNs work effectively in many application domains, it may suffer from a problemcalled vanishing gradients [81]. To cope with this problem, two variants of RNN has been developed:Long Short-Term Memory (LSTM) [58] and Gated Recurrent Units (GRU) networks [23]. LSTMis capable of learning long-term dependencies with a special memory unit as shown in Fig 5. AnLSTM cell has three gates (forget gate, input gate, and output gate) to regulate the informationflow. Forget gate decides which information wefire going to remember in the cell state. Input gatedecides what new information wefire going to store and output gate decides what informationwefire going to output. Compared with standard LSTM models, GRU has fewer parameters whichcombines the input gate and the forget gate into an ’update gate’ and merges the cell state andhidden state. RNN, LSTM and GRU are widely used to learn the temporal correlations of time seriesand spatio-temporal data.
Autoencoder (AE) [57] is a neural network that is trained to copy itsinput to its output by learning data codings in an unsupervised manner [45]. The network iscomposed of two parts: encoder and decoder as shown in Fig. 6. Encoder function h = f ( x ) compresses the input into a latent-space representation and decoder r = д ( h ) reconstructs the inputthrough the representation. Autoencoder can learn the useful properties of the input data and iscommonly used for dimensionality reduction, feature learning, and generative modelling. As acommonly used unsupervised representation learning method, AE is popular for classification andprediction tasks in trajectories [106, 164], time series [61] and other spatio-temporal data [31]. The Graph Convolutional Network (GCN) is capableof extracting representations from hidden layers which encode both local graph structure andnode features and it is claimed to be linearly scalable with the size of graph [70]. Traffic prediction
ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020. becomes popular in recent years and several spatio-temporal deep learning models based on GCNshave been proposed. Yu et al. introduced Spatio-Temporal Graph Convolutional Networks (STGCN)[155] to solve the prediction problem in traffic networks. Normal deep learning models have someissues in dealing with spatio-temporal forecasting tasks, such as heavy computation of trainingin RNN-based networks and the normal convolutional operation is limited on grid structures. Tosolve these problems, STGCN converts traffic network into the graph-structured format and useseveral spatio-temporal convolutional blocks to learn spatial and temporal dependencies. Eachof the blocks consists of graph convolutional layers and convolutional sequence learning layerswhich reduce the cost of computation via approximation strategies such as Chebyshev PolynomialsApproximation or First order Approximation. To seize the spatial and temporal dependenciessimultaneously, Zhao et al. also model traffic network via graph and solve traffic forecasting taskswith a novel model called the Temporal Graph Convolutional Network (T-GCN) [160]. In thismodel, the GCN is applied to learn complex topological structures for extracting spatial dependenceand the gated recurrent unit (GRU) is responsible to learn dynamic dependence at temporal aspect.Recently, the Graph Multi-Attention Network (GMAN) [162] and the Attention-based Spatial-Temporal Graph Convolutional Network (ASTGCN) [52] all adapt attention mechanisms withGCN models to learn the impact of the spatio-temporal factors on traffic conditions. GMANhas an encoder to extract the traffic features as input and predicts the output sequence by thedecoder. Several ST-Attention blocks are deployed in both encoder and decoder, and each blockcontains a spatial attention, a temporal attention and a gated fusion for modelling the correlationsbetween vertices and time frames. Besides, one transform attention layer is used as an intermediatecomponent to reduce the error propagation effect. Differently, ASTGCN uses three independentcomponents to model hourly- periodic, daily-periodic and weekly-periodic dependencies fromtraffic flows, respectively. However, it is similar to GMAN, each component has an attention toeffectively capture the dynamic spatial- temporal correlations in traffic flow and then conductconvolution on the constructed network with GCN.GCN further demonstrates its capability for human action recognition. Yan el at. model dynamicsof human body skeletons via graphs to retain information for human actions [150]. In this work,they propose a novel variant of Spatial-Temporal Graph Convolutional Networks (ST-GCN), whichautomatically learns both the spatial and temporal patterns from human actions data. The skeletonsequences of human actions are represented by a spatial-temporal graph in a hierarchical way,which contains N human joints and T frames and features not only intra-skeleton connection butalso the links between same joints between consecutive frames. To construct the convolutionalnetworks on the defined skeleton graph, the CNN filters are designed for the convolution operationon both the neighbours of one node within one single frame and those across consecutive timeframes. In this section, we will introduce the basic idea of generative adversarial nets. Then, we will discussthe popular GAN variants and loss functions especially used in the spatio-temporal data modellingapplications.
The original concept of GANs is to create two neural networks and let them compete against eachother. As shown in Figure 7, the basic architecture of GAN comprises two components: a generatorand a discriminator. On the one hand, the generator’s task is to synthesis fake images which can
ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020. enerative Adversarial Networks for Spatio-temporal Data: A Survey 111:9 G Vanilla GAN
D G D
CGAN
G D
InfoGAN
G D
ACGAN
G D
Semi-Supervised GAN
Fig. 7. A view of variants of GAN. G represents the generator network, D is the discriminator network, z represents the noise, c means the class labels, x f are the generated fake images and x r are the real images fool the discriminator. On the other hand, the discriminator, as to its name, learns to distinguish ifits input is a fake image or not [46].If we left the images generation task aside, the underlying idea of generative adversarial netsis more general, which is to create one fake distribution p д and make it as close as possible to adata distribution p r . The reason we use such an approach is that p r could be hard to get directlyand by doing in this manner, we get a good approximation of it and then we can sample from thisapproximate distribution instead [8]. The advantages of this approach are that since the generatoris learning to approximate the real distribution directly, there is no need to introduce the MarkovChain and no inference is required due to the isolation between the generator and the real datadistribution. Besides, its simple structure makes it easier to incorporate with other techniques[100].The Generator G ( z ; θ д ) , a neural network that parameterized by theta takes a sample z ∼ p ( z ) as input and mapping that to a sample x ∼ p д . And its rival, the Discriminator D ( x ; θ d ) outputs asingle binary value that indicates its prediction of realness. During the training session, both partsare trained simultaneously and based on their opponent’s result, which forms a minimax gamewith the overall objective function [46]: min G max D V ( D , G ) = E x ∼ p data ( x ) [ log D ( x )] + E z ∼ p z ( z ) [ log ( − D ( G ( z )))] Despite all the aforementioned advantages, the original generative adversarial network is stillinadequate in some places. The practical results show that the training is particularly delicate andthe generators may suffer from mode dropping/collapse [8]. To address all those problems mightoccur, many variants of the vanilla GAN are proposed.In this paper, we divided those popular variants of GAN into two different categories: modificationof its structure and changes on the loss function . The former one is focusing on improving itsperformance by increasing the overall complexity of its structure while the latter one pays attentionto the deficiency of the Jensenfi?! Shannon divergence. Since the plain MLP (multi-layer preceptrons)generator and discriminators in the vanilla GAN [46] can be seamlessly replaced by other types ofneural networks such as CNN [117], we do not consider them as the modification of the structure.
ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020.
And it should also be noted that adding auxiliary parts normally leads to a change in the lossfunction but as long as it still keeps the JSD, it would not be treated as a new type of loss functionin this paper.
Although the vanilla GAN shows its potential for data generation [46], and the discriminator inthis structure is proved to be effective on classification task [117]. But it still suffers from blurryand possible mode dropping/collapse. Besides, there is no control in the generation process sinceits unsupervised manner [108]. To this end, some researches introduce other machine learningtechniques into the original GAN structure, and some results are promising. In this section, wedescribe three types of the variant in that direction.
Support Info . Mirza et al. [100] proposed CGAN (Conditional GAN) which introduces a supportinfo vector y . In the generator, each input z gets its corresponding y , and it is also available thediscriminator which can help it for better judgement. Since this vector is a controlled parameterrather than another random sample, we gain some level of control of the samples generated. Chenet al. [19], on the other hand, is also focused on providing support info to the generator, andproposed the InfoGAN. A latent code c is adding to the input of the generator, however, instead ofletting the discriminator do all the works, InfoGAN calculates the mutual information I ( c ; G ( z , c )) to indicate the level of info remains after the generation process. By maximising this regularisationterm, the result generator can be controlled according to the latent code c . Odena et al. [108]introduced a supervised task into the original GAN and proposed ACGAN (Auxiliary ClassifierGAN). Every sample from the real data belongs to a predefined class, and an expected label c ∼ p c along with noise z ∼ p z is used as input to generate a data sample of that class. Besides the real/fakediscrimination task, an auxiliary classifier is created to classify every sample. This enables theability for the generator to synthesis sample for a particular class. Hierarchical Structure . Zhang et al. [159] proposed StackGAN (Stacked GAN) to cope withthe problems that the generated images are blurry and hard to scale up. In this structure, two GANsare created, and each of them having different tasks. The task of its first layer, or also called ”Stage-I”GAN is to generate low-resolution images with primitive shapes and colours, while the ”Stage-II”GAN is used as a refiner to increase the resolution to the desired level and corrects possible defects.It also incorporates the CGAN into it, since its task is to synthesis images based on a given sentence.On the base of that idea, Juefei-Xu et al. makes a further improvement and proposed GoGAN(Gang of GANs) [66]. In this method, the whole structure is divided into multiple ranking stage,and each stage has its own GAN model. However, unlike StackGAN, which gives different tasks togenerators at different stages, all generators in the GoGAN have the same task and same input.A unique constraint is applied to enforce that the images generated by the later stages should becloser to the real data compared to their ancestors. This enables competitions in more dimensionsexcept for the generator versus discriminator one described in the vanilla GAN, and analysis showsthat it provides faster convergence than WGAN.Except getting multiple sets of generator and discriminator, Karras et al. proposed ProGAN(Progressive growing of GANs) [68] which utilise the same GAN structure and creating moredetailed images by incrementally adding more layers to the existing generator. To avoid the damagethat could backpropagate from the newly added layer, it uses a weight addition between theupsampling result from the last layer and the image from the new layer. And this new layer willsmoothly fade in by introducing a monotone increasing hyperparameter. Although it has the abilityto generate images with larger pixels and finer details, this progressive training does consume a lotof computational power and the deeper it goes, the more consuming it will be.
ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020. enerative Adversarial Networks for Spatio-temporal Data: A Survey 111:11
In traditional generative modelling approaches, the performance of a modelis indicated by the Kullback-Leibler (KL) divergence between our desire distribution p r and ourgenerator’s distribution p д [8]. D KL ( P r (cid:107) P д ) = ∫ χ P r ( x ) log P r ( x ) P д ( x ) d x Minimising this term means making those two distributions closer, and it would get to zeroonce p r = p д . However, the generator might still generate fake-looking data due to the imbalancednature [8] of this function. It could heavily penalise the generator for the part that is in the realdistribution but not covered by the generator while paying less attention to the extra part coveredby the generator. In order to avoid this weakness, the loss function of the discriminator in theoriginal GAN paper is switched by a balanced version called Jensen-Shannon (JS) divergence [46]. D J S ( P r (cid:107) P д ) = D KL (cid:0) P r (cid:107) ( P r + P д ) (cid:1) + D KL (cid:0) P д (cid:107) ( P r + P д ) (cid:1) Although it shows some promising results, JS divergence is not the ultimate choice since it stillsuffers from issues like mode collapsing. Some late research shows that those can be resolved byusing other types of loss function [9, 97, 107]. In this subsection, we describe two major types ofloss functions and the GAN method based on them. f -divergence . As mentioned in the previous section, aside from the competitive structure, onedifference between GAN and traditional generative modelling methods is the use of JS divergenceinstead of KL divergence. Its balanced characteristic makes it more suitable for machine learningmodels to optimise, but it still prunes to mode dropping/collapse empirically [8, 107].Instead of settling down on just a single divergence metric, Nowozin et al. [107] proposed f -GAN,which allows us to choose from many other metrics by introducing a term called f -divergence. D f ( P (cid:107) Q ) = ∫ χ q ( x ) f (cid:18) p ( x ) q ( x ) (cid:19) d x Choosing different f function can lead to different divergence metrics such as KL, reserve KL andSquared-Hellinger divergence. And by using a convex conjugate function, also known as Fenchelconjugate , we can switch the original GAN’s objective function by the following equation: F ( θ , ω ) = E x ∼ P [ T ω ( x )] − E x ∼ Q θ [ f ∗ ( T ω ( x ))] where, P is the real distribution, Q θ is the approximate distribution controlled by parameter θ , and T ω is the variational function that serves as our discriminator.Mao et al. [97] proposed LSGAN and in that paper, the author dully discussed the relationbetween their choice of the objective function and f -divergence and showed that it could beequivalently minimising the Pearson χ divergence which makes LSGAN a special form of f -GAN. Integral Probability Metric . Integral Probability Metric (IPM) is another family of metrics thatcould be used to measure the distance between two certain distributions [104]. Some metrics inthis type show nicer properties compared to the original JS Divergence used in vanilla GAN suchas less oscillation in the generator training. We mainly cover Wasserstein GAN (WGAN) [9] andits improved version [50] in this paper since they are the most commonly-used ones.Arjovsky et al. [9] proposed Wasserstein GAN, which including the
Earth-Mover (EM) distanceor
Wasserstein-1 distance shows below: W ( P r , P д ) = inf γ ∈ (cid:206) ( P r , P д ) E ( x , y )∼ γ [(cid:107) x − y (cid:107)] ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020.
This metric indicates the amount of ”dirt” needs to be moved in order to transform a distribution P into another Q . This is much suitable than the Jensen-Shannon divergence since it provides someexcellent characteristics to counter the weakness of original GAN. Firstly, since the EM distanceis changing continuously with less huge variations, the gradients of the generator would not beconstant zero with an optimal or sub-optimal discriminator. This solves the vanishing gradientsproblem in training the generator. Secondly, it could also help to cope with mode dropping problemsince the EM distance only reaches zero if those two distributions are the same.However, there are still some vulnerabilities in this method. The infimum in the original EMequation is highly intractable and have to be converted to the following form: W ( P r , P д ) = sup (cid:107) f (cid:107) L ≤ E x ∼ P r [ f ( x )] − E x ∼ P д [ f ( x )] where the (cid:107) f (cid:107) L ≤ K -Lipschitz for some constant K ). To enforce this constraint, the author uses weightclipping as a preliminary approach [9] and improved to gradient penalty, which shows fasterconvergence and allows the use of momentum optimiser like Adam optimiser [50]. In this section, we propose a taxonomy of GANs for spatio-temporal data and modelling tasks. Asillustrated in Table. 1, the formation of our taxonomy mainly comes from several aspects, e.g., STdatatypes and tasks. GANs for spatio-temporal events prediction is firstly introduced. We thendiscuss tasks for sequence modelling with time series, including sequence generation, imputationand prediction. Recent GAN architectures for graph data are also reviewed. To be specific, we focuson two major tasks for graph data: link prediction and graph representation or embedding. Besides,we discuss some recent work on the trajectory prediction task which has become a popular topic inthe research community. Based on this taxonomy, we review the recent progress of applying GANsfor different types of spatio-temporal data in the following subsections. In addition, in Table 2, wesummarise widely used datasets for each type of spatio-temporal data.
In this subsection, we will mainly introduce how GANs are applied to predict the spatio-temporalevents (e.g., taxi demand [130, 156], crime [64], fluid flows [21], anomaly detection [79]) in thefuture based on the previous events.For the first time, Saxena et al. [130] proposed a generative adversarial network
D-GAN foraccurate spatio-temporal events prediction. In the model, GAN and VAE are combined to jointlylearn generation and variational inference of ST data in an unsupervised manner. They alsodesigned a general fusion module to fuse heterogeneous multiple data sources. Figure 8 shows thearchitecture for D-GAN, consisting of four components:
Encoder , Generator/Decoder , Discriminator ,and
External feature fusion . G network is trained using the adversarial process in which decoder (i.e.,generator) learns to approximate the distribution of real data, while the D network discriminatebetween samples generated by D and samples from real distributions. During the training process,D-GAN adopts a reconstruction loss and adversarial loss [130]. In addition, ConvLSTM [149] and
3D -ConvNet structures were exploited to model long-term patterns and spatial dependencies in STdata.Recently, Yu et al. [156] applied a conditional generative adversarial network with long short-term structure (LSTM-CGAN) for taxi hotspot prediction, which captures the spatial and temporalvariations of hotspots simultaneously. Jin et al. [64] developed a context-based generative model
Crime-GAN to learn the spatio-temporal dynamics of crime situation. They aggregated Seq2Seq,
ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020. enerative Adversarial Networks for Spatio-temporal Data: A Survey 111:13 S T m ap s E x t e r na l f ea t u r e m ap s ... Other external feature maps
Xrealf C o n v L S T M a n d D C o n v N e t C o n v L S T M a n d D C o n v N e t MLPMLP MeanStd. dev.MeanStd. dev.
Concatenate Sampled latent vector of XrealSampled latent vector of f MLP NoiseDeep fused representation C o n v L S T M a n d D C o n v N e t X r e a l X f a k e X e n e Real?Fake?Encoder Generator DiscriminatorFeature extraction Dense feature fusion Feature restoration C o n v L S T M a n d D C o n v N e t Fig. 8. D-GAN architecture proposed by Seaena et al. [130]
VAE network and adversarial loss in the framework to better study ST data representation. Fur-thermore, the deep convolutional generative adversarial network (DCGAN) has been developed forspatio-temporal fluid flow prediction in a tsunami case in Japan [21].GANs have also been used for anomaly detection for spatio-temporal events. Li et al. [79]proposed MAD-GAN, an unsupervised anomaly detection for multivariate time series based on GAN.They trained a GAN generator and discriminator with LSTM; Then, the GAN-trained generator anddiscriminator are employed to detect anomalies in the testing data with a combined Discriminationand Reconstruction Anomaly Score (DR-Score).
The applications of GAN technique on sequence data mainly focus on two aspects: generation andimputation. We will discuss them separately as follows.
Data generation refers to creating data from the sampled data source. One ofthe main purposes of time series generation with GAN is to protect the privacy of sensitive datasuch as medical data [33], electroencephalographic (EEG) data [55], heart signal electrocardiogram(ECG) data [44], occupancy data [20], electronic health records (EHR) [18], etc.Recently, GANs have been used to generate sequential data. Mogren et al. [101] proposedC-RNN-GAN (continuous RNN-GAN) to generate continuous-valued sequential data. They builtthe GAN with LSTM generator and discriminator, and the discriminator consists of a bidirectionallayout which allows it to take context in both directions into account for its decisions. They trainedthe model on sequences of classical music and evaluated with metrics such as polyphony, scaleconsistency, repetitions and tone span.Then, Esteban et al. [33] proposed a regular GAN where both the generator and the discriminatorhave been substituted by recurrent neural networks. They presented the Recurrent GAN (RGAN)and Recurrent Conditional GAN (RCGAN) to generate sequences of real-valued medical data ordata subject to some conditional inputs. For evaluation, they proposed to use the capability of the
ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020. T a b l e . R e v i e w o f S p a t i o - t e m p o r a l D a t aa n d M o d e ll i n g T a s k s u s i n g G A N S T d a t a t y p e R e f e r e n c e Y e a r T a s k V a l u e s i n t a s k M o d e l c a t e g o r y E v a l u a t i o n m e t h o d s T i m e s e r i e s C - R NN - G A N [ ] G e n e r a t i o n M u s i c a l d a t a G A N a n d L S T M D o m a i n m e t r i c s ( e . g ., p o l y p h o n y , s c a l e c o n s i s t e n c y , r e p e t i t i o n s , t o n e s p a n ) R C G A N [ ] G e n e r a t i o n M e d i c a l d a t a G A N a n d R NN T S T R a n d T R T SSS L - G A N [ ] G e n e r a t i o n E l e c t r o n i c h e a l t h r e c o r d s G A N , C NN a n d A E P r e d i c t i o n a cc u r a c y o v e r m u l t i p l e d a t a c o m b i n a t i o n s O cc u G A N [ ] G e n e r a t i o n O cc u p a n c y d a t a G A N a n d D NN D o m a i n m e t r i c s ( e . g ., t i m e o f fi r s t a rr i v a l , n u m b e r o f o cc u p i e d t r a n s i t i o n s ) G r i d - G A N [ ] G e n e r a t i o n S m a r t g r i dd a t a C G A N a n d C NN T S T R a n d T R T S EE G - G A N [ ] G e n e r a t i o n EE G b r a i n s i g n a l s W G A N a n d C NN I S , F I D a n d E D S t o c k G A N [ ] G e n e r a t i o n S t o c k d a t a G A N , C NN a n d L S T M P r e d i c t i o n a cc u r a c y ( e . g ., R M S R E , D P A ) G R U - G A N [ ] I m p u t a t i o n M e d i c a l r e c o r d s , m e t e o r o l o g i c d a t a G A N a n d G R U I m p u t a t i o n a cc u r a c y F o r G A N [ ] G e n e r a t i o n S y n t h e t i c s e r i e s a n d i n t e r n e tt r a ffi c C G A N a n d L S T M K L d i v e r g e n c e N A O M I [ ] I m p u t a t i o n T r a ffi c fl o w , m o v e m e n t d a t a G A N a n d R NN I m p u t a t i o n a cc u r a c y T i m e G A N [ ] G e n e r a t i o n S i n e s , s t o c k s , e n e r g y a n d e v e n t s d a t a G A N a n d A E D i v e r s i t y , fi d e l i t y a n d u s e f u l n e ss ( e . g ., T S T R ) E G A N [ ] I m p u t a t i o n M e d i c a l r e c o r d s , m e t e o r o l o g i c d a t a G A N a n d G R U I m p u t a t i o n a cc u r a c y S i m G A N [ ] G e n e r a t i o n H e a r t r a t e E C G s i g n a l s G A N P r e d i c t i o n a cc u r a c y o v e r m u l t i p l e G A N m e t h o d s A d - A tt a c k [ ] G e n e r a t i o n S t o c k p r i c e s a n d e l e c t r i c i t y d a t a G A N D o m a i n m e t r i c s ( e . g , a tt a c k s u c e ss r a t e , r e t u r n e d o f p e r t u r b e d p o r t f o l i o ) A O S R e c [ ] G e n e r a t i o n S e q u e n c e s o f r e c o mm e n d a t i o n G A N a n d G R U P r e c i s i o n , n D C G a n d B L E U T r a j e c t o r y G D - G A N [ ] P r e d i c t i o n P e d e s t r a i n t r a j e c t o r i e s G A N a n d L S T M A v e r a g e d i s p l a c e m e n t e rr o r ( A D E ) a n d fi n a l d i s p l a c e m e n t e rr o r ( F D E ) S o c i a l G A N [ ] P r e d i c t i o n S o c a i ll y a cc e p t a b l e t r a j e c t o r i e s G A N a n d L S T M Qu a n t i t a t i v e ( e . g ., A D E , F D E ) a n d q u a l i t a t i v e ( e . g ., g r o u p a v o i d i n g ) m e t r i c s S o P h i e [ ] P r e d i c t i o n P e d e s t r a i n t r a j e c t o r i e s G A N a n d L S T M A D E a n d F D E S o c i a l W a y s [ ] P r e d i c t i o n P e d e s t r a i n t r a j e c t o r i e s G A N a n d L S T M A D E a n d F D E S o c i a l - B i G A T [ ] P r e d i c t i o n P e d e s t r a i n t r a j e c t o r i e s G A N a n d L S T M A D E a n d F D E A P O I R [ ] P r e d i c t i o n P o i n t - o f - I n t e r e s t s G A N a n d G R U P r e c i s i o n , R e c a ll , n D C G a n d M A P C o L - G A N [ ] P r e d i c t i o n P e d e s t r a i n t r a j e c t o r i e s G A N , C NN a n d L S T M A v e r a g e c o ll i s i o n t i m e s ( A C T ) , A D E a n d F D E A d a tt T U L [ ] L i n k p r e d i c t i o n H u m a n m o b i l i t y t r a j e c t o r i e s G A N , G R U a n d L S T M P r e d i c t i o n a cc u r a c y o v e r m u l t i p l e m o d e l s S T e v e n t s D - G A N [ ] P r e d i c t i o n T a x i a n d b i k e d a t a G A N a n d V A E P r e d i c t i o n a cc u r a c y o v e r m u l t i p l e m o d e l s T a x i - C G A N [ ] P r e d i c t i o n T a x i h o t s p o t s d a t a C G A N a n d L S T M F a l s e i d e n t i fi c a t i o n t e s t ( F I T ) a n d t h e s e c t i o n c o n s i s t e n c y t e s t ( S C T ) C r i m e - G A N [ ] P r e d i c t i o n C r i m e d a t a D C G A N , C NN a n d R NN P r e d i c t i o n a cc u r a c y ( e . g ., J S d i v e r g e n c e ) o v e r m u l t i p l e m o d e l s M A D - G A N [ ] P r e d i c t i o n C y b e r - a tt a c k s d a t a G A N a n d L S T M D R - s c o r e G r a p h s G r a p h G A N [ ] R e p r e s e n t a t i o n S o c i a l n e t w o r k s G A N a n d D NN P r e d i c t i o n a cc u r a c y o v e r m u l t i p l e m o d e l s N e t G A N [ ] R e p r e s e n t a t i o n C i t a t i o n a n d b l o g s n e t w o r k s W G A N a n d L S T M P r e d i c t i o n a cc u r a c y o v e r m u l t i p l e m o d e l s A N E [ ] R e p r e s e n t a t i o n C i t a t i o n a n d b l o g s n e t w o r k s G A N a n d D NN P r e d i c t i o n a cc u r a c y o v e r m u l t i p l e m o d e l s N e t R A [ ] R e p r e s e n t a t i o n S o c i a l a n d b i o l o g i c a l n e t w o r k s G A N , L S T M a n d A E P r e d i c t i o n a cc u r a c y o v e r m u l t i p l e m o d e l s G C N - G A N [ ] L i n k P r e d i c t i o n M o b i l i t y n e t w o r k s G A N , G C N a n d L S T MM S E , e d g e - w i s e K L d i v e r g e n c e , m i s m a t c h r a t e G A N E [ ] R e p r e s e n t a t i o n C o a u t h o r n e t w o r k s W G A N a n d D NN P r e d i c t i o n a cc u r a c y o v e r m u l t i p l e m o d e l s N e t w o r k G A N [ ] L i n k P r e d i c t i o n S o c i a l n e t w o r k s G A N , G C N a n d L S T M R M S E , A U C , K L d i v e r g e n c e P r o G A N [ ] R e p r e s e n t a t i o n S o c i a l a n d c i t a t i o nn e t w o r k s G A N a n d D NN P r e d i c t i o n a cc u r a c y o v e r m u l t i p l e m o d e l s M E G A N [ ] R e p r e s e n t a t i o n S o c i a l m u l t i - v i e w n e t w o r k s G A N a n d M L PP r e d i c t i o n a cc u r a c y o v e r m u l t i p l e m o d e l s ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020. enerative Adversarial Networks for Spatio-temporal Data: A Survey 111:15
Table 2. Summary of Datasets
ST data type Dataset Data source Used references
Time series
Philips eICU database [114] Medical data [33]MNIST [148] Hand-written digit images [33]Occupancy dataste [84] Occupancy data in the building [20]Pecan street dataset [65] Energy consumption, solar generation [158]PhysioNet dataset [136] Medical data (e.g., heart rate, glucose) [44, 92, 93]KDD cup 2018 dataset [2] Air quality data [92, 93]A5M dataset [24] Transatlantic link data [72]PEMS-SF traffic dataset [30] Freeway occupancy rate [91]Appliances energy dataset [17] Environmental data [154]UCI electricity dataset [141] Historical price data [26]Yoochoose [14] Clicking events from users [161]MovieLens [54] Movie ratings data [161]
Trajectory
ETH [112] Videos [7, 34, 53, 73, 88, 126]UCY [78] Videos [7, 34, 53, 73, 88, 126]Stanford drone dataset [121] Videos [126]Vittorio emanuele II [11] Videos [34]Foursquare [151] Location-based social networks [40, 80, 90, 96, 165]Gowalla [22] Location-based social networks [40, 90, 165]Brightkite [22] Location-based social networks [40, 96]Yelp [4] Location-based social networks [80, 96, 165]
ST events
Yellow taxi dataset [3] Taxi demand data [120]CitiBike trip dataset [103] Bike demand data [120]SWaT dataset [43] Attacked data in water system [79]
Graphs
ArXiv-AstroPh [1] Scientific collaborations data [143]Wikipedia [49] Network of words [25, 143, 157]CORA [98] Citation networks of publications [15, 25, 36]CiteSeer [132] Citation networks of publications [15, 25, 36]DBLP [110] Collaboration graph of authors [15, 25, 152, 157]Blogcatalog [139] Social network for bloggers [36, 143, 157]UCI message dataset [109] Message communication networks [152, 157]Flickr [122] Social networks [36, 138] generated synthetic data to train supervised models, i.e., TSTR (train on synthetic, test on real).They addressed that TSTR is more effective than TRTS (train on real, test on synthetic) becauseTRTS performance may not degrade when GAN suffers mode collapse.GANs have been used for the generation of biological-physiological signals such as EEG andECG. Hartmann et al. [55] proposed EEG-GAN to generate electroencephalographic (EEG) brainsignals. With the modification of the improved WGAN training, they trained a GAN to produceartificial signals in a stable fashion which strongly resembles single-channel real EEG signals in thetime and frequency domain. For evaluation metrics, they showed that the combination of Frechetinception distance (FID) and sliced Wasserstein distance (SWD), Euclidean distance (ED) can givea good idea about its overall properties. Golany et al. [44] proposed a simulator-based GANs forECG synthesis to improve a supervised classification. They incorporated ECG simulator equationsinto the generation networks, and then the generated ECG signals are used train a deep network.
ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020.
Chen et al. [20] proposed GAN framework for building occupancy modelling. They first learnedthe discriminator and generator in the vanilla GAN with the training occupancy data. Then, thelearned generator is the required occupancy model which can be used to generate occupancydata with random inputs. To evaluate, they defined five variables (i.e., mean occupancy, timeof the first arrival, time of the last departure, cumulative occupied duration and the number ofoccupied/unoccupied transitions) with two criteria (i.e., normalised root mean squared error andtotal variation distance).Che et al. [18] used a modified GAN called ehrGAN to generate plausible labelled EHR data. Thegenerator is a modified encoder-decoder CNN network and the generated EHR data mimics thereal patient records which augments the training dataset in a semi-supervised learning manner. Inthis work, they used the generative networks with the CNN prediction model together to improvethe performance of risk prediction.Koochali et al. [72] proposed
ForGAN to predict the next-step time series value X t + by learningthe full conditional probability distribution. They applied a conditional GAN and the conditionwindows are the previous t values ( X , X , ..., X t ). With the input of noise vector, the generatorpredicts the values at t + t + y t + based on thefeatures in previous t time step X , X , ..., X t and previous stock price y , y , ..., y t using generativeadversarial nets.Instead of generating a sequence of single values, Dang et al. [26] developed an approach forthe generation of adversarial attacks where the output is a sequence of probability distributions.The proposed approaches are demonstrated on two challenging tasks including the predictionof electricity consumption and stock market trading. Besides, AOSeRec [161] were proposed togenerate a sequence of items consistent with user preferences rather than the next-item prediction.The model integrated the sequence-level oracle and adversarial learning into the seq2seq auto-regressive learning.Generally, a good time-series generative model should preserve temporal dynamics, and thegenerated sequences should follow the original patterns between variables across time. Therefore,Yoon et al. [154] proposed a framework TimeGAN for producing realistic multivariate time-series,combining the flexibility of the unsupervised GAN approach with the control afforded by supervisedlearning. In addition to the traditional unsupervised adversarial loss on both real and fake data,they presented a stepwise supervised loss with the original data as supervision, which helps learnfrom the transition dynamics in real sequences.
In real-world applications, time series are usually incomplete due to variousreasons, and the time intervals of observations are usually not fixed [92]. The missing valuesin time series make it hard for effective analysis [41]. One of the popular ways to handle themissing values of time series is to impute the missing values to get the complete dataset. Generally,there are three different ways for time series imputation: case deletion methods [67], statisticalimputation methods [47], and machine learning based imputation methods [13]. However, all theexisting approaches hardly consider the temporal relations between two observations. In recentyears, researchers have started to take advantages of GANs to learn latent representations betweenobservations for time series imputation [91–93].Luo et al. [92] applied the adversarial model to generate and impute the original incomplete timeseries. To learn the latent relationships between observations with non-fixed time lags, a novelRNN cell called GRUI was proposed which takes into account the non-fixed time lags and fadesthe influence of the past observations determined by the time lags. They proposed a two-stage
ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020. enerative Adversarial Networks for Spatio-temporal Data: A Survey 111:17
DiscriminatorReal Incomplete time series Generated complete Time series P(real)Gradient FeedbackGenerator
Generate
Random noise
Fig. 9. An overview of the time series imputation framework proposed by Luo et al. [92] model (see Figure 9) for time series imputation: In the first stage, they adopted the GRUI in thediscriminator and generator in GAN to learn the distribution and temporal information of thedataset. In the second stage, for each sample, they tried to optimise the ’noise’ input vector andfind the best-matched input vector of the generator. The noise was trained with a two-part lossfunction: masked reconstruction loss and discriminative loss. Masked reconstruction loss is themasked squared errors of the non-missing part between the original and generated sample. It meansthat the generated time series should be close enough to the original incomplete time series. Thediscriminative loss forces the generated sample as real as possible. However, this two-stage modelneeds a huge time to find the best-matched input vector which is not always the best especiallywhen the initial value of the ’noise’ is not properly set.Then, Luo et al. [93] proposed an end-to-end GAN-based imputation model E GAN which notonly simplifies the process of time series imputation but also generates more reasonable values forthe filling of missing values. E GAN takes a compressing and reconstructing strategy to avoid the’noise’ optimisation stage in [92]. As seen in Fig. 10, in the generator (a denoising auto-encoder),they added a random vector to the original sample and map it into a low-dimensional vector. Thenthey reconstructed it from the low-dimensional vector. The generator seeks to find a networkstructure that can not only best compress and reconstruct the multivariate time series but alsofools the discriminator. Then they used the reconstructed sample to impute the missing values.Non-Autoregressive Multiresolution Imputation (NAOMI) [91] is a new model for the imputationof spatio-temporal sequences like traffic flow data and movement trajectories when arbitrarymissing observations are given. NAOMI impute missing values for spatio-temporal sequencesrecursively from coarse to fine-grained resolutions with a non-autoregressive decoding procedure,and it further employs a generative adversarial learning process to reduce variance for improvingthe performance.
In this subsection, we will introduce the application of GAN on the graph data analysis whichmainly focus on two areas: temporal link prediction and graph representation.
Temporal link prediction refers to the dynamics predictionproblem in network systems (e.g., mobility and traffic prediction) where system behaviours aredescribed by the abstract graphs [77]. Given the snapshots of a graph in previous timestamps, thetemporal link prediction task aims to construct the graph topology at the next timestamp. Lei et al.
ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020.
DiscriminatorIncomplete time series AGenerated complete Time series A' Z P(real)Incomplete time series A Gradient Feedback
Fig. 10. An overview of E GAN framework proposed by Luo et al. [93] [77] proposed GCN-GAN to predict links in weighted dynamic networks. They combined graphconvolutional network (GCN), long short-term memory (LSTM) as well as generative adversarialnetwork (GAN). The generator consists of a GCN hidden layer, LSTM hidden layer and a fully-connected layer. Discriminator contains a fully-connected feed-forward network. For evaluation,they used edge-wise KL divergence and mismatch rate besides mean square error (MSE). Then, Yanget al. [152] designed an attentive GCN model for temporal link prediction in graphs using GAN.Compared to [77], attentive GCN allows for assigning different importance to the vertices to learnthe spatial features of the dynamic network. Then, temporal matrix factorisation (TMF) LSTM wasemployed to capture the temporal dependencies and evolutionary patterns of dynamic networks.GAN framework was then proposed to improve the performance of temporal link prediction.
Wang et al. [143] proposed GraphGAN unifying two types of graphrepresentation methods: discriminative methods and generative methods via adversarial training.They found that the traditional softmax function and its variants are not suitable for the generatorfor two reasons: 1) softmax treats all vertices equally in the graph for a given vertex and does notconsider the graph structure and proximity information; 2) the calculation of softmax involves allvertices in the graph which is time-consuming and computationally inefficient. Therefore, theyintroduced graph softmax as the implementation of the generator and proved that it satisfies thedesirable properties of normalisation, computational efficiency and graph structure awareness.Aiming at better capturing the essential properties and preserving the patterns of real graphs,Bojchevski et.al. introduced NetGAN [15] to learn a distribution of network via the random walks.The merits of using random walks is their invariance under node reordering and efficiency inexploring the sparse graphs by merely traversing the nonzero entries. The results confirmed thatthe combination of longer random walks and LSTM is advantageous for the model to learn thetopology and general patterns in the data.Adversarial Network Embedding (ANE) [25] also considers random walk mechanism to learnnetwork representation with the adversarial learning principle. It consisted of two components:1) the structure-preserving component is developed to extract network structural properties viathe usage of either Inductive DeepWalk or Denoising Autoencoder; 2) the adversarial learning
ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020. enerative Adversarial Networks for Spatio-temporal Data: A Survey 111:19 component contributes to learning network representations by matching the posterior distribu-tion of the latent representations to given priors. However, using DeepWalk for learning graphembedding could lead to overfitting issue due to sparsity is common in networks or increasingcomputational burden when more sampled walks are considered [157]. Therefore, NetRA [157]was proposed to further minimise network locality-preserving loss and global reconstruction errorwith a discrete LSTM Autoencoder and continuous space generator, in such that the mapping frominput sequences into vertex representations could be improved.Most recently, GAN embedding (GANE) [59] tries to gain the underlying graph distribution basedon the probability distribution of edge existence which is similar to GraphGAN. The difference is thatthis model applies Wasserstein-1 distance as the overall objective function and intents to achievelink prediction and network embedding extraction simultaneously. As a novel network embeddingmethod, the proximity generative adversarial network (ProGAN) [36] is proposed to capture theunderlying proximity between different nodes by approximating the generated distribution ofnodes in a triplet format to the underlying proximity in the model of GAN. Specifically, a tripletcan encode the relationship among three nodes including similarity and dissimilarity. After thetraining of the generator and discriminator, the underlying proximities discovered are then used tobuild network embedding with an encoder.The works mentioned above primarily focus on the single-view network in learning networkembedding. However, numerous real-world data are represented by multi-view networks whosenodes have different types of relations. Sun et.al. [138] introduced a new framework for multi-viewnetwork embedding called MEGAN, which can preserve the information from individual networkviews, while considering nodes connectivity within one relation and complex correlations amongdifferent views. During the training of MEGAN, a pair of nodes are chosen from the generator basedon the fake connectivity pattern across views which is produced by multi-layer perceptron (MLP),and the discriminator is then executed to differentiate the real pair of nodes from the generatedone.
Trajectory prediction refers to the problem of estimating the future trajectories of various agentsbased on the previous observations [95]. Gupta et al. [53] proposed SocialGAN to jointly predicttrajectories avoiding collisions for all people. They introduced a variety loss encouraging thegenerative network of the GAN to spread its distribution and cover the space of possible pathswhile being consistent with the observed inputs. A new pooling mechanism was proposed to learna figlobalfi pooling vector which encodes the subtle cues for all people involved in a scene. InGD-GAN [34], Fernando et al. designed a GAN based pipeline to jointly learn features for bothpedestrian trajectory prediction and social group detection. As the basic GAN structure used inSocialGAN is susceptible to mode collapsing and dropping issues, Amirian et al. [7] extended theSocialGAN by incorporating the Info-GAN [19] structure in their
Social Ways trajectory predictionnetwork.
SoPhie , proposed by Sadeghian et al. [126], is another GAN based trajectory prediction approachwhich can take both the information from the scene context and social interactions of the agentsinto consideration. Two separate attention modules are also used to better capture the scenecontext and the social interactions. More recently, based on BicycleGAN [167] framework, Social-BiGAT [73] develops the bijection function between the output trajectories and the latent spaceinput to the trajectory generator. It also uses a Graph Attention Network in combination with aVGG network [137] to encode social influence from other pedestrians and semantic scene influenceof the environment. In order to generate trajectories with fewer potential collisions, CoL-GAN [88],
ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020. proposed by Liu et al., exploits a CNN-based network as the trajectory discriminator. Differentfrom other GAN based trajectory prediction methods such as SocialGAN [53] and SoPhie [126],the proposed discriminator is able to classify whether each segment of a trajectory is real or fake.Recently, Gao et al. [40] studied the trajectory user linking problem to identify user identitiesfrom mobility patterns. They combined autoencoder with GANs for jointly human mobility learning,which provides regularized latent space for mobility classification. APOIR [165] were developed tolearn the distribution of underlying user preferences in the Point-of-interest (POI) recommendation.It consists of two components: the recommender and discriminator. The recommender approachesthe true preference of users and the discriminator distinguishes the generated POIs from the trulyvisited ones.
Alongside numerous advantages of GANs, there are still challenges needed to be solved for em-ploying GANs in ST applications. The traditional architectures and loss functions of GANs maynot be suitable due to the unique properties of ST data. Besides, evaluating ST data is more difficultcompared to images where researchers could rely on the visual inspections. Therefore, we willmainly focus on: (1) how to modify architectures/loss functions of GANs to better capture the spatialand temporal relations for ST data? (2) how to evaluate the performance of GANs especially whenvisually inspecting the generated ST samples is not applicable?
We will then address these twoproblems and indicate the future directions of investigating this area.
Architectures and loss functions of GANs . In the computer vision area, fully connected layerswere initially used as building blocks in vanilla GAN, but later on were replaced by convolutionallayers in DCGAN [117]. Compared with images with only spatial relations, modelling ST datais more complex due to the constraints from both spatial and temporal dimensions. Therefore,adapting architectures and loss functions of GANs for specific ST applications have become themainstream recently.Generally, original or adapted RNN [33, 92, 101] , LSTM [15, 72, 77, 79, 157], VAE [18, 93, 130, 157],CNN [18], GNN [77] are usually used as the base model (i.e., the discriminator and generator) inthe vanilla GAN , WGAN [55] or CGAN [72], which captures the spatio-temporal relations for thespatio-temporal data. What’s more, some new loss functions have been proposed to dealing withspecific ST tasks, such as the stepwised supervised loss in TimeGAN [154], masked reconstructionloss in GRU-GAN [92], the variety loss in SocialGAN [7]. With further developments of GANs forST data, new architectures and loss functions can be designed based on the characteristics of STtasks.
Evaluation Metrics . Though GANs have gained huge success in various fields, evaluating theperformance of GANs is still an open question. As illustrated in [16] and [60], both quantitativelymeasures (e.g.,
Log-likelihood with Parzen Window Estimation [129],
Frchet Inception Distance [56],
Maximum Mean Discrepancy [48]) and qualitative measures (e.g.,
Preference Judgement [144],Analysing
Internals of Models [117]) have strengths and limitations. The nebulous notion of qualitycan be best assessed by a human judge, which is neither practical nor appropriate for differenttypes of ST data.In most cases, it is not easy or even possible to visually evaluate the generated ST data. Forinstance, the
Intense Care Unit (ICU) time series [33] or heart rate
Electrocardiogram (ECG) [44]signals could look completely random to a non-medical expert. Usually, the evaluation of generatedST samples requires domain knowledge. For example, Mogren et al. [101] evaluated the generatedmusic sequences using metrics in the field of music such as polyphony, repetitions, tone span and
ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020. enerative Adversarial Networks for Spatio-temporal Data: A Survey 111:21 scale consistency. For future ST applications with GANs, some novel metrics based on the domainknowledge could be considered for the evaluation of generated ST data.Especially, some researchers have proposed the general approach to evaluate the generatedST-data. Esteban et al. [33] developed a general method called
Train on Synthetic, Test on Real (TSTR) to evaluate the generated samples of GANs when a supervised task defined on the trainingdata. They used a dataset generated by GANs to train a classification model, which is then testedon a held-out set of true samples. This evaluation metric is ideal when employing GANs to sharesynthetic de-identified data because it demonstrates the ability of the generated synthetic data tobe used for real applications. In the future, more practical metrics should be developed to evaluatethe performance of generated ST samples.
In this survey, we conducted a comprehensive overview of
Generative Adversarial Networks (GANs)for spatio-temporal data in recent years. Firstly, we discussed the properties of spatio-temporaldata and traditional ways for spatio-temporal data modelling. Then, we have provided a thoroughreview and comparison of the popular variants of GANs, and its applications on spatio-temporaldata analysis, such as time series imputation, trajectory prediction, graph representation and linkprediction. Besides, we summarised the challenges and future directions for employing GANs forspatio-temporal applications.Finally, we would like to point out, though there are many promising results in the literature,the adoption of GANs for spatio-temporal data is still in its infancy. This survey can be used asthe stepping stone for future research in this direction, which provides a detailed explanation ofdifferent spatio-temporal applications with GANs. We wish this paper could help readers identifythe set of problems and choose the relevant GAN techniques when given a new spatio-temporaldataset.
This research was supported by the Australian Government through the Australian Research Coun-cil’s Linkage Projects funding scheme (project LP150100246) and Discovery Project (DP190101485).
REFERENCES
Information Systems
53 (2015), 16–38.[6] Husam Al Waer and Mark Deakin. 2012.
From intelligent to smart cities . Routledge.[7] Javad Amirian, Jean-Bernard Hayet, and Julien Pettr´e. 2019. Social ways: Learning multi-modal distributions ofpedestrian trajectories with GANs. In
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops .[8] M Arjovsky and L Bottou. [n.d.]. Towards principled methods for training generative adversarial networks.
Interna-tional Conference on Learning Representations (ICLR) .[9] Martin Arjovsky, Soumith Chintala, and L´eon Bottou. 2017. Wasserstein gan. arXiv preprint arXiv:1701.07875 (2017).[10] Gowtham Atluri, Anuj Karpatne, and Vipin Kumar. 2018. Spatio-temporal data mining: A survey of problems andmethods.
ACM Computing Surveys (CSUR)
51, 4 (2018), 1–41.[11] Stefania Bandini, Andrea Gorrini, and Giuseppe Vizzari. 2014. Towards an integrated approach to crowd analysisand crowd synthesis: A case study and first results.
Pattern Recognition Letters
44 (2014), 16–29.[12] Bj¨orn Barz, Erik Rodner, Yanira Guanche Garcia, and Joachim Denzler. 2018. Detecting regions of maximal divergencefor spatio-temporal anomaly detection.
IEEE Transactions on Pattern Analysis and Machine Intelligence
41, 5 (2018),1088–1101. ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020. [13] Gustavo EAPA Batista and Maria Carolina Monard. 2003. An analysis of four missing data treatment methods forsupervised learning.
Applied artificial intelligence
17, 5-6 (2003), 519–533.[14] David Ben-Shimon, Alexander Tsikinovsky, Michael Friedmann, Bracha Shapira, Lior Rokach, and Johannes Hoerle.2015. Recsys challenge 2015 and the yoochoose dataset. In
Proceedings of the 9th ACM Conference on RecommenderSystems . 357–358.[15] Aleksandar Bojchevski, Oleksandr Shchur, Daniel Z¨ugner, and Stephan G¨unnemann. 2018. Netgan: Generatinggraphs via random walks.
International Conference on Machine Learning (ICML) (2018).[16] Ali Borji. 2019. Pros and cons of gan evaluation measures.
Computer Vision and Image Understanding
179 (2019),41–65.[17] Luis M Candanedo, V´eronique Feldheim, and Dominique Deramaix. 2017. Data driven prediction models of energyuse of appliances in a low-energy house.
Energy and buildings
140 (2017), 81–97.[18] Zhengping Che, Yu Cheng, Shuangfei Zhai, Zhaonan Sun, and Yan Liu. 2017. Boosting deep learning risk predictionwith generative adversarial networks for electronic health records. In
IEEE International Conference on Data Mining(ICDM) . IEEE, 787–792.[19] Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. 2016. Infogan: Interpretablerepresentation learning by information maximizing generative adversarial nets. In
Advances in Neural InformationProcessing Systems . 2172–2180.[20] Zhenghua Chen and Chaoyang Jiang. 2018. Building occupancy modeling using generative adversarial network.
Energy and Buildings
174 (2018), 372–379.[21] M Cheng, Fangxin Fang, Christopher C Pain, and IM Navon. 2020. Data-driven modelling of nonlinear spatio-temporalfluid flows using a deep convolutional generative adversarial network.
Computer Methods in Applied Mechanics andEngineering
365 (2020), 113000.[22] Eunjoon Cho, Seth A Myers, and Jure Leskovec. 2011. Friendship and mobility: user movement in location-basedsocial networks. In
Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and datamining . 1082–1090.[23] Kyunghyun Cho, Bart Van Merri¨enboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the properties of neuralmachine translation: Encoder-decoder approaches.
The Conference on Empirical Methods in Natural LanguageProcessing (EMNLP) Workshops (2014).[24] Paulo Cortez, Miguel Rio, Miguel Rocha, and Pedro Sousa. 2012. Multi-scale Internet traffic forecasting using neuralnetworks and time series methods.
Expert Systems
29, 2 (2012), 143–155.[25] Quanyu Dai, Qiang Li, Jian Tang, and Dan Wang. 2018. Adversarial network embedding. In
Thirty-Second AAAIConference on Artificial Intelligence .[26] Rapha¨el Dang-Nhu, Gagandeep Singh, Pavol Bielik, and Martin Vechev. 2020. Adversarial Attacks on ProbabilisticAutoregressive Forecasting Models.
International Conference on Machine Learning (ICML) (2020).[27] Bertrand De Longueville, Robin S Smith, and Gianluca Luraschi. 2009. Omg, from here, i can see the flames!: ause case of mining location based social networks to acquire spatio-temporal data on forest fires. In
InternationalWorkshop on Location Based Social Networks . ACM, 73–80.[28] Shohreh Deldari, Daniel V Smith, Amin Sadri, and Salim Flora D. 2020. ESPRESSO: Entropy and ShaPe awaRetimE-Series SegmentatiOn for processing heterogeneous sensor data.
Proceedings of the ACM on Interactive, Mobile,Wearable and Ubiquitous Technologies
4, 3 (2020).[29] Chris Donahue, Zachary C Lipton, Akshay Balsubramani, and Julian McAuley. 2018. Semantically decomposing thelatent spaces of generative adversarial networks.
International Conference on Learning Representations (ICLR) .[30] Dheeru Dua and Casey Graff. 2017. UCI machine learning repository. (2017).[31] Yanjie Duan, Yisheng Lv, Wenwen Kang, and Yifei Zhao. 2014. A deep learning based approach for traffic dataimputation. In . IEEE, 912–917.[32] Kiana Ehsani, Roozbeh Mottaghi, and Ali Farhadi. 2018. Segan: Segmenting and generating the invisible. In
IEEEConference on Computer Vision and Pattern Recognition (CVPR) . 6144–6153.[33] Crist´obal Esteban, Stephanie L Hyland, and Gunnar R¨atsch. 2017. Real-valued (medical) time series generation withrecurrent conditional gans. arXiv preprint arXiv:1706.02633 (2017).[34] Tharindu Fernando, Simon Denman, Sridha Sridharan, and Clinton Fookes. 2018. Gd-gan: Generative adversarialnetworks for trajectory prediction and group detection in crowds. In
Asian Conference on Computer Vision . Springer,314–330.[35] Elias Frentzos, Yannis Theodoridis, and Apostolos N. Papadopoulos. 2009.
Spatio-Temporal Trajectories . Springer US,Boston, MA, 2742–2746. https://doi.org/10.1007/978-0-387-39940-9 364[36] Hongchang Gao, Jian Pei, and Heng Huang. 2019. ProGAN: Network Embedding via Proximity Generative AdversarialNetwork. In
Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining .1308–1316.ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020. enerative Adversarial Networks for Spatio-temporal Data: A Survey 111:23 [37] Nan Gao, Wei Shao, Mohammad Saiedur Rahaman, and Flora D Salim. 2020. n-Gage: Predicting in-class Emotional,Behavioural and Cognitive Engagement in the Wild.
Proceedings of the ACM on Interactive, Mobile, Wearable andUbiquitous Technologies
4, 3 (2020).[38] Nan Gao, Wei Shao, Mohammad Saiedur Rahaman, Jun Zhai, Klaus David, and Flora D Salim. 2020. Transfer Learningfor Thermal Comfort Prediction in Multiple Cities. arXiv preprint arXiv:2004.14382 (2020).[39] Nan Gao, Wei Shao, and Flora D Salim. 2019. Predicting personality traits from physical activity intensity.
Computer
52, 7 (2019), 47–56.[40] Qiang Gao, Fengli Zhang, Fuming Yao, Ailing Li, Lin Mei, and Fan Zhou. 2020. Adversarial Mobility Learning forHuman Trajectory Classification.
IEEE Access
Computers inbiology and medicine
59 (2015), 125–133.[42] Betsy George, Sangho Kim, and Shashi Shekhar. 2007. Spatio-temporal network databases and routing algorithms: Asummary of results. In
International Symposium on Spatial and Temporal Databases . Springer, 460–477.[43] Jonathan Goh, Sridhar Adepu, Khurum Nazir Junejo, and Aditya Mathur. 2016. A dataset to support research in thedesign of secure water treatment systems. In
International Conference on Critical Information Infrastructures Security .Springer, 88–99.[44] Tomer Golany, Daniel Freedman, and Kira Radinsky. 2020. SimGANs: Simulator-Based Generative AdversarialNetworks for ECG Synthesis to Improve Deep ECG Classification.
International Conference on Machine Learning(ICML) .[45] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016.
Deep learning . MIT press.[46] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, andYoshua Bengio. 2014. Generative adversarial nets. In
Advances in Neural Information Processing Systems . 2672–2680.[47] John W Graham. 2009. Missing data analysis: Making it work in the real world.
Annual review of psychology
Journal of Machine Learning Research
13, Mar (2012), 723–773.[49] Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In
Proceedings of the 22ndACM SIGKDD international conference on Knowledge discovery and data mining . 855–864.[50] Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. 2017. Improved trainingof wasserstein gans. In
Advances in Neural Information Processing Systems . 5767–5777.[51] Venkata M. V. Gunturi, Ivan Brugere, and Shashi Shekhar. 2018.
Modeling and Analysis of Spatiotemporal SocialNetworks . Springer New York, New York, NY, 1358–1368. https://doi.org/10.1007/978-1-4939-7131-2 320[52] Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. 2019. Attention based spatial-temporal graphconvolutional networks for traffic flow forecasting. In
Proceedings of the AAAI Conference on Artificial Intelligence ,Vol. 33. 922–929.[53] Agrim Gupta, Justin Johnson, Li Fei-Fei, Silvio Savarese, and Alexandre Alahi. 2018. Social gan: Socially acceptabletrajectories with generative adversarial networks. In
IEEE Conference on Computer Vision and Pattern Recognition(CVPR) . 2255–2264.[54] F Maxwell Harper and Joseph A Konstan. 2015. The movielens datasets: History and context.
Acm transactions oninteractive intelligent systems (tiis)
5, 4 (2015), 1–19.[55] Kay Gregor Hartmann, Robin Tibor Schirrmeister, and Tonio Ball. 2018. EEG-GAN: Generative adversarial networksfor electroencephalograhic (EEG) brain signals. arXiv preprint arXiv:1806.01875 (2018).[56] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. Gans trainedby a two time-scale update rule converge to a local nash equilibrium. In
Advances in Neural Information ProcessingSystems . 6626–6637.[57] Geoffrey E Hinton and Ruslan R Salakhutdinov. 2006. Reducing the dimensionality of data with neural networks. science
Neural computation
9, 8 (1997), 1735–1780.[59] Huiting Hong, Xin Li, and Mingzhong Wang. 2019. GANE: A Generative Adversarial Network Embedding.
IEEETransactions on Neural Networks and Learning Systems (2019).[60] Yongjun Hong, Uiwon Hwang, Jaeyoon Yoo, and Sungroh Yoon. 2019. How Generative Adversarial Networks andTheir Variants Work: An Overview.
ACM Computing Surveys (CSUR)
52, 1 (2019), 10.[61] Moinul Hossain, Banafsheh Rekabdar, Sushil J Louis, and Sergiu Dascalu. 2015. Forecasting the weather of Nevada:A deep learning approach. In
International Joint Conference on Neural Networks (IJCNN) . IEEE, 1–6.[62] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-image translation with conditionaladversarial networks. In
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 1125–1134.ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020. [63] Sadegh Jamali, Per J¨onsson, Lars Eklundh, Jonas Ard¨o, and Jonathan Seaquist. 2015. Detecting changes in vegetationtrends using time series segmentation.
Remote Sensing of Environment
156 (2015), 182–195.[64] Guangyin Jin, Qi Wang, Xia Zhao, Yanghe Feng, Qing Cheng, and Jincai Huang. 2019. Crime-GAN: A Context-basedSequence Generative Network for Crime Forecasting with Adversarial Loss. In
IEEE International Conference on BigData (Big Data) arXiv preprint arXiv:1704.04865 (2017).[67] Jiri Kaiser. 2014. Dealing with missing values in data.
Journal of systems integration
5, 1 (2014), 42–51.[68] Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2018. Progressive growing of gans for improved quality,stability, and variation.
International Conference on Learning Representations (ICLR) .[69] Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXivpreprint arXiv:1609.02907 (2016).[70] Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks.
Interna-tional Conference on Learning Representations (ICLR) .[71] Slava Kisilevich, Florian Mansmann, Mirco Nanni, and Salvatore Rinzivillo. 2009. Spatio-temporal clustering. In
Datamining and knowledge discovery handbook . Springer, 855–874.[72] Alireza Koochali, Peter Schichtel, Andreas Dengel, and Sheraz Ahmed. 2019. Probabilistic Forecasting of SensoryData With Generative Adversarial Networks–ForGAN.
IEEE Access
Advances inNeural Information Processing Systems . 137–146.[74] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neuralnetworks. In
Advances in Neural Information Processing Systems . 1097–1105.[75] Matthew Kupilik and Frank Witmer. 2018. Spatio-temporal violent event prediction using Gaussian process regression.
Journal of Computational Social Science
1, 2 (2018), 437–451.[76] Christian Ledig, Lucas Theis, Ferenc Husz´ar, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken,Alykhan Tejani, Johannes Totz, Zehan Wang, et al. 2017. Photo-realistic single image super-resolution using agenerative adversarial network. In
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 4681–4690.[77] Kai Lei, Meng Qin, Bo Bai, Gong Zhang, and Min Yang. 2019. GCN-GAN: A non-linear temporal link predictionmodel for weighted dynamic networks. In
IEEE INFOCOM 2019-IEEE Conference on Computer Communications . IEEE,388–396.[78] Alon Lerner, Yiorgos Chrysanthou, and Dani Lischinski. 2007. Crowds by example. In
Computer graphics forum ,Vol. 26. Wiley Online Library, 655–664.[79] Dan Li, Dacheng Chen, Baihong Jin, Lei Shi, Jonathan Goh, and See-Kiong Ng. 2019. MAD-GAN: Multivariateanomaly detection for time series data with generative adversarial networks. In
International Conference on ArtificialNeural Networks . Springer, 703–716.[80] Ruirui Li, Xian Wu, and Wei Wang. 2020. Adversarial Learning to Compare: Self-Attentive Prospective CustomerRecommendation in Location based Social Networks. In
Proceedings of the 13th International Conference on WebSearch and Data Mining . 349–357.[81] Shuai Li, Wanqing Li, Chris Cook, Ce Zhu, and Yanbo Gao. 2018. Independently recurrent neural network (indrnn):Building a longer and deeper rnn. In
Proceedings of the IEEE conference on computer vision and pattern recognition .5457–5466.[82] Xuefei Li, Hongyun Cai, Zi Huang, Yang Yang, and Xiaofang Zhou. 2013. Spatio-temporal event modeling andranking. In
International Conference on Web Information Systems Engineering . Springer, 361–374.[83] Zhenhui Li, Bolin Ding, Jiawei Han, Roland Kays, and Peter Nye. 2010. Mining periodic behaviors for moving objects.In
Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining . ACM,1099–1108.[84] Chenda Liao, Yashen Lin, and Prabir Barooah. 2012. Agent-based and graphical modelling of building occupancy.
Journal of Building Performance Simulation
5, 1 (2012), 5–25.[85] T Warren Liao. 2005. Clustering of time series datafi!?a survey.
Pattern recognition
38, 11 (2005), 1857–1874.[86] Kevin Lin, Dianqi Li, Xiaodong He, Zhengyou Zhang, and Ming-Ting Sun. 2017. Adversarial ranking for languagegeneration. In
Advances in Neural Information Processing Systems . 3155–3165.[87] Ming-Yu Liu and Oncel Tuzel. 2016. Coupled generative adversarial networks. In
Advances in Neural InformationProcessing Systems . 469–477.[88] S. Liu, H. Liu, H. Bi, and T. Mao. 2020. CoL-GAN: Plausible and Collision-less Trajectory Prediction by Attention-basedGAN.
IEEE Access (2020), 1–10.ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020. enerative Adversarial Networks for Spatio-temporal Data: A Survey 111:25 [89] Siyuan Liu, Lionel M Ni, and Ramayya Krishnan. 2013. Fraud detection from taxis’ driving behaviors.
IEEE Transactionson Vehicular Technology
63, 1 (2013), 464–472.[90] Wei Liu, Zhi-Jie Wang, Bin Yao, and Jian Yin. 2019. Geo-ALM: POI Recommendation by Fusing GeographicalInformation and Adversarial Learning Mechanism.. In
IJCAI . 1807–1813.[91] Yukai Liu, Rose Yu, Stephan Zheng, Eric Zhan, and Yisong Yue. 2019. NAOMI: Non-autoregressive multiresolutionsequence imputation. In
Advances in Neural Information Processing Systems . 11238–11248.[92] Yonghong Luo, Xiangrui Cai, Ying Zhang, Jun Xu, et al. 2018. Multivariate time series imputation with generativeadversarial networks. In
Advances in Neural Information Processing Systems . 1596–1607.[93] Yonghong Luo, Ying Zhang, Xiangrui Cai, and Xiaojie Yuan. 2019. E2GAN: end-to-end generative adversarial networkfor multivariate time series imputation. In
International Joint Conference on Artificial Intelligence (IJCAI) . AAAI Press,3094–3100.[94] Teng Ma, Yuli Liu, and Yuejing Ge. 2017. A comparative study of trade relations and the spatial-temporal evolutionof geo-economy between China and Vietnam.
Sustainability
9, 6 (2017), 944.[95] Yuexin Ma, Xinge Zhu, Sibo Zhang, Ruigang Yang, Wenping Wang, and Dinesh Manocha. 2019. Trafficpredict:Trajectory prediction for heterogeneous traffic-agents. In
Proceedings of the AAAI Conference on Artificial Intelligence ,Vol. 33. 6120–6127.[96] Jarana Manotumruksa and Emine Yilmaz. 2020. Sequential-based Adversarial Optimisation for Personalised Top-NItem Recommendation. In
Proceedings of the 43rd International ACM SIGIR Conference on Research and Development inInformation Retrieval . 2045–2048.[97] Xudong Mao, Qing Li, Haoran Xie, Raymond YK Lau, Zhen Wang, and Stephen Paul Smolley. 2017. Least squaresgenerative adversarial networks. In
IEEE International Conference on Computer Vision (ICCV) . 2794–2802.[98] Andrew McCallum, Kamal Nigam, and Lyle H Ungar. 2000. Efficient clustering of high-dimensional data sets withapplication to reference matching. In
Proceedings of the sixth ACM SIGKDD international conference on Knowledgediscovery and data mining . 169–178.[99] Tom´aˇs Mikolov, Martin Karafi´at, Luk´aˇs Burget, Jan ˇCernock`y, and Sanjeev Khudanpur. 2010. Recurrent neuralnetwork based language model. In
Eleventh annual conference of the international speech communication association .[100] Mehdi Mirza and Simon Osindero. 2014. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014).[101] Olof Mogren. 2016. C-RNN-GAN: Continuous recurrent neural networks with adversarial training. arXiv preprintarXiv:1611.09904 (2016).[102] Steffen Moritz and Thomas Bartz-Beielstein. 2017. imputeTS: time series missing value imputation in R.
The R Journal
Advances in AppliedProbability
29, 2 (1997), 429–443.[105] K-R M¨uller, Alexander J Smola, Gunnar R¨atsch, Bernhard Sch¨olkopf, Jens Kohlmorgen, and Vladimir Vapnik. 1997.Predicting time series with support vector machines. In
International Conference on Artificial Neural Networks . Springer,999–1004.[106] Nam Tuan Nguyen, Yichuan Wang, Husheng Li, Xin Liu, and Zhu Han. 2012. Extracting typical users’ movingpatterns using deep learning. In . IEEE, 5410–5414.[107] Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. 2016. f-gan: Training generative neural samplers usingvariational divergence minimization. In
Advances in Neural Information Processing Systems . 271–279.[108] Augustus Odena, Christopher Olah, and Jonathon Shlens. 2017. Conditional image synthesis with auxiliary classifiergans. In
Proceedings of the 34th International Conference on Machine Learning-Volume 70 . JMLR. org, 2642–2651.[109] Tore Opsahl and Pietro Panzarasa. 2009. Clustering in weighted networks.
Social networks
31, 2 (2009), 155–163.[110] Shirui Pan, Jia Wu, Xingquan Zhu, Chengqi Zhang, and Yang Wang. 2016. Tri-party deep network representation.
Network
11, 9 (2016), 12.[111] Zhaoqing Pan, Weijie Yu, Xiaokai Yi, Asifullah Khan, Feng Yuan, and Yuhui Zheng. 2019. Recent progress ongenerative adversarial networks (GANs): A survey.
IEEE Access
European conference on computer vision (ECCV) . Springer, 452–465.[113] Karthik Ganesan Pillai, Rafal A Angryk, and Berkay Aydin. 2013. A filter-and-refine approach to mine spatiotemporalco-occurrences. In
Proceedings of the 21st ACM SIGSPATIAL International Conference on Advances in GeographicInformation Systems . 104–113.[114] Tom J Pollard, Alistair EW Johnson, Jesse D Raffa, Leo A Celi, Roger G Mark, and Omar Badawi. 2018. The eICUCollaborative Research Database, a freely available multi-center database for critical care research.
Scientific data [115] Arian Prabowo, Piotr Koniusz, Wei Shao, and Flora D Salim. 2019. COLTRANE: ConvolutiOnaL TRAjectory NEtworkfor Deep Map Inference. In
Proceedings of the 6th ACM International Conference on Systems for Energy-EfficientBuildings, Cities, and Transportation . ACM, 21–30.[116] Kyle K Qin, Wei Shao, Yongli Ren, Jeffrey Chan, and Flora D Salim. 2019. Solving multiple travelling officers problemwith population-based optimization algorithms.
Neural Computing and Applications (2019), 1–27.[117] Alec Radford, Luke Metz, and Soumith Chintala. 2015. Unsupervised representation learning with deep convolutionalgenerative adversarial networks. arXiv preprint arXiv:1511.06434 (2015).[118] Mohammad Saiedur Rahaman, Margaret Hamilton, and Flora D Salim. 2017. Predicting Imbalanced Taxi and PassengerQueue Contexts in Airport.. In
The Pacific Asia Conference on Information Systems (PACIS) . 172.[119] Mohammad Saiedur Rahaman, Jonathan Liono, Yongli Ren, Jeffrey Chan, Shaw Kudo, Tim Rawling, and Flora DSalim. 2020. An Ambient-Physical System to Infer Concentration in Open-plan Workplace.
IEEE Internet of ThingsJournal (2020).[120] Yongli Ren, Flora Dilys Salim, Martin Tomko, Yuntian Brian Bai, Jeffrey Chan, Kyle Kai Qin, and Mark Sanderson.2017. D-Log: A WiFi Log-based differential scheme for enhanced indoor localization with single RSSI source andinfrequent sampling rate.
Pervasive and Mobile Computing
37 (2017), 94–114.[121] Alexandre Robicquet, Amir Sadeghian, Alexandre Alahi, and Silvio Savarese. 2016. Learning social etiquette: Humantrajectory understanding in crowded scenes. In
European conference on computer vision (ECCV) . Springer, 549–565.[122] Ryan A. Rossi and Nesreen K. Ahmed. 2015. The Network Data Repository with Interactive Graph Analytics andVisualization. In
AAAI . http://networkrepository.com[123] Frantz Rowe. 2014. What literature review is not: diversity, boundaries and recommendations.[124] Shakila Khan Rumi, Ke Deng, and Flora Dilys Salim. 2018. Crime event prediction with dynamic features.
EPJ DataScience
7, 1 (2018), 43.[125] Shakila Khan Rumi, Phillip Luong, and Flora D Salim. 2019. Crime Rate Prediction with Region Risk and MovementPatterns. arXiv preprint arXiv:1908.02570 (2019).[126] Amir Sadeghian, Vineet Kosaraju, Ali Sadeghian, Noriaki Hirose, Hamid Rezatofighi, and Silvio Savarese. 2019.SoPhie: An attentive gan for predicting paths compliant to social and physical constraints. In
IEEE Conference onComputer Vision and Pattern Recognition (CVPR) . 1349–1358.[127] Amin Sadri, Flora Dilys Salim, and Yongli Ren. 2017. Full trajectory prediction: what will you do the rest of the day?.In
Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedingsof the 2017 ACM International Symposium on Wearable Computers . ACM, 189–192.[128] Amin Sadri, Flora D Salim, Yongli Ren, Wei Shao, John C Krumm, and Cecilia Mascolo. 2018. What will you do forthe rest of the day?: An approach to continuous trajectory prediction.
Proceedings of the ACM on Interactive, Mobile,Wearable and Ubiquitous Technologies
2, 4 (2018), 186.[129] Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. 2016. Improvedtechniques for training gans. In
Advances in Neural Information Processing Systems . 2234–2242.[130] Divya Saxena and Jiannong Cao. 2019. D-GAN: Deep Generative Adversarial Nets for Spatio-Temporal Prediction. arXiv preprint arXiv:1907.08556 (2019).[131] Johannes Schmidt-Hieber. 2017. Nonparametric regression using deep neural networks with ReLU activation function. arXiv preprint arXiv:1708.06633 (2017).[132] Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. 2008. Collectiveclassification in network data.
AI magazine
29, 3 (2008), 93–93.[133] Wei Shao, Flora D Salim, Jeffrey Chan, Kai Qin, Jiaman Ma, and Bradley Feest. 2019. OnlineAirTrajClus: AnOnline Aircraft Trajectory Clustering for Tarmac Situation Awareness. In
IEEE International Conference on PervasiveComputing and Communications (PerCom . IEEE, 192–201.[134] Wei Shao, Flora D Salim, Tao Gu, Ngoc-Thanh Dinh, and Jeffrey Chan. 2017. Traveling officer problem: managing carparking violations efficiently using sensor data.
IEEE Internet of Things Journal
5, 2 (2017), 802–810.[135] Wei Shao, Flora D Salim, Andy Song, and Athman Bouguettaya. 2016. Clustering big spatiotemporal-interval data.
IEEE Transactions on Big Data
2, 3 (2016), 190–203.[136] Ikaro Silva, George Moody, Daniel J Scott, Leo A Celi, and Roger G Mark. 2012. Predicting in-hospital mortality of icupatients: The physionet/computing in cardiology challenge 2012. In . IEEE, 245–248.[137] Karen Simonyan and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition.In
International Conference on Learning Representations (ICLR) .[138] Yiwei Sun, Suhang Wang, Tsung-Yu Hsieh, Xianfeng Tang, and Vasant Honavar. 2019. Megan: A generative adversarialnetwork for multi-view network embedding.
International Joint Conference on Artificial Intelligence (IJCAI) .[139] Lei Tang and Huan Liu. 2009. Relational learning via latent social dimensions. In
Proceedings of the 15th ACM SIGKDDinternational conference on Knowledge discovery and data mining . 817–826.[140] Steven A Tretter. 1976.
Introduction to discrete-time signal processing . Wiley New York.ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020. enerative Adversarial Networks for Spatio-temporal Data: A Survey 111:27 [141] Artur Trindade. 2016. Uci maching learning repository-electricityloaddiagrams20112014 data set.[142] Carl Vondrick, Hamed Pirsiavash, and Antonio Torralba. 2016. Generating videos with scene dynamics. In
Advancesin Neural Information Processing Systems . 613–621.[143] Hongwei Wang, Jia Wang, Jialin Wang, Miao Zhao, Weinan Zhang, Fuzheng Zhang, Xing Xie, and Minyi Guo. 2018.Graphgan: Graph representation learning with generative adversarial nets. In
Thirty-Second AAAI Conference onArtificial Intelligence .[144] Jifeng Wang, Xiang Li, and Jian Yang. 2018. Stacked conditional generative adversarial networks for jointly learningshadow detection and shadow removal. In
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) .1788–1797.[145] Senzhang Wang, Jiannong Cao, and Philip S Yu. 2019. Deep learning for spatio-temporal data mining: A survey. arXiv preprint arXiv:1906.04928 (2019).[146] Zhengwei Wang, Qi She, and Tomas E Ward. 2019. Generative adversarial networks in computer vision: A surveyand taxonomy. arXiv preprint arXiv:1906.01529 (2019).[147] Andreas S Weigend. 2018.
Time series prediction: forecasting the future and understanding the past . Routledge.[148] Han Xiao, Kashif Rasul, and Roland Vollgraf. 2017. Fashion-mnist: a novel image dataset for benchmarking machinelearning algorithms. arXiv preprint arXiv:1708.07747 (2017).[149] SHI Xingjian, Zhourong Chen, Hao Wang, Dit-Yan Yeung, Wai-Kin Wong, and Wang-chun Woo. 2015. ConvolutionalLSTM network: A machine learning approach for precipitation nowcasting. In
Advances in Neural InformationProcessing Systems . 802–810.[150] Sijie Yan, Yuanjun Xiong, and Dahua Lin. 2018. Spatial temporal graph convolutional networks for skeleton-basedaction recognition. In
Thirty-second AAAI conference on artificial intelligence .[151] Dingqi Yang, Daqing Zhang, Vincent W Zheng, and Zhiyong Yu. 2014. Modeling user activity preference by leveraginguser spatial temporal characteristics in LBSNs.
IEEE Transactions on Systems, Man, and Cybernetics: Systems
45, 1(2014), 129–142.[152] Min Yang, Junhao Liu, Lei Chen, Zhou Zhao, Xiaojun Chen, and Ying Shen. 2019. An Advanced Deep GenerativeFramework for Temporal Link Prediction in Dynamic Networks.
IEEE Transactions on Cybernetics (2019).[153] Xin Yi, Ekta Walia, and Paul Babyn. 2019. Generative adversarial network in medical imaging: A review.
Medicalimage analysis
58 (2019), 101552.[154] Jinsung Yoon, Daniel Jarrett, and Mihaela van der Schaar. 2019. Time-series generative adversarial networks. In
Advances in Neural Information Processing Systems . 5508–5518.[155] Bing Yu, Haoteng Yin, and Zhanxing Zhu. 2018. Spatio-temporal graph convolutional networks: a deep learningframework for traffic forecasting. In
International Joint Conference on Artificial Intelligence (IJCAI) . 3634–3640.[156] Hao Yu, Zhenning Li, Guohui Zhang, Pan Liu, and Jun Wang. 2020. Extracting and Predicting Taxi Hotspots inSpatiotemporal Dimensions Using Conditional Generative Adversarial Neural Networks.
IEEE Transactions onVehicular Technology
69, 4 (2020), 3680–3692.[157] Wenchao Yu, Cheng Zheng, Wei Cheng, Charu C Aggarwal, Dongjin Song, Bo Zong, Haifeng Chen, and Wei Wang.2018. Learning deep network representations with adversarially regularized autoencoders. In
Proceedings of the 24thACM SIGKDD International Conference on Knowledge Discovery & Data Mining . 2663–2671.[158] Chi Zhang, Sanmukh R Kuppannagari, Rajgopal Kannan, and Viktor K Prasanna. 2018. Generative adversarialnetwork for synthetic time series data generation in smart grids. In
IEEE International Conference on Communications,Control, and Computing Technologies for Smart Grids (SmartGridComm) . IEEE, 1–6.[159] Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, and Dimitris N Metaxas. 2017.Stackgan: Text to photo-realistic image synthesis with stacked generative adversarial networks. In
IEEE internationalConference on Computer Vision (ICCV) . 5907–5915.[160] Ling Zhao, Yujiao Song, Chao Zhang, Yu Liu, Pu Wang, Tao Lin, Min Deng, and Haifeng Li. 2019. T-gcn: A temporalgraph convolutional network for traffic prediction.
IEEE Transactions on Intelligent Transportation Systems (2019).[161] Pengyu Zhao, Tianxiao Shui, Yuanxing Zhang, Kecheng Xiao, and Kaigui Bian. 2020. Adversarial Oracular Seq2seqLearning for Sequential Recommendation. In
Proceedings of the Twenty-Ninth International Joint Conference onArtificial Intelligence, IJCAI 2020 , Christian Bessiere (Ed.). ijcai.org, 1905–1911. https://doi.org/10.24963/ijcai.2020/264[162] Chuanpan Zheng, Xiaoliang Fan, Cheng Wang, and Jianzhong Qi. 2020. Gman: A graph multi-attention network fortraffic prediction. In
Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 34. 1234–1241.[163] Yu Zheng. 2015. Trajectory data mining: an overview.
ACM Transactions on Intelligent Systems and Technology (TIST)
6, 3 (2015), 29.[164] Fan Zhou, Qiang Gao, Goce Trajcevski, Kunpeng Zhang, Ting Zhong, and Fengli Zhang. 2018. Trajectory-UserLinking via Variational AutoEncoder.. In
International Joint Conference on Artificial Intelligence (IJCAI) . 3212–3218.[165] Fan Zhou, Ruiyang Yin, Kunpeng Zhang, Goce Trajcevski, Ting Zhong, and Jin Wu. 2019. Adversarial point-of-interestrecommendation. In
The World Wide Web Conference . 3462–34618.ACM Computing Surveys, Vol. 37, No. 4, Article 111. Publication date: August 2020. [166] Xingyu Zhou, Zhisong Pan, Guyu Hu, Siqi Tang, and Cheng Zhao. 2018. Stock market prediction on high-frequencydata using generative adversarial nets.
Mathematical Problems in Engineering