Data-driven Analysis for Understanding Team Sports Behaviors
aa r X i v : . [ c s . A I] F e b Machine Learning based Analysis for Team Sports Behaviors
Paper:
Data-driven Analysis for Understanding Team Sports Behaviors
Keisuke Fujii
Nagoya University, Furocho 1, Nagoya, Aichi, 464-8603, JAPANE-mail: [email protected]
Understanding the principles of real-world biologicalmulti-agent behaviors is a current challenge in var-ious scientific and engineering fields. The rules re-garding the real-world biological multi-agent behav-iors such as team sports are often largely unknowndue to their inherently higher-order interactions, cog-nition, and body dynamics. Estimation of the rulesfrom data, i.e., data-driven approaches such as ma-chine learning, provides an effective way for the anal-ysis of such behaviors. Although most data-drivenmodels have non-linear structures and high predictionperformances, it is sometimes hard to interpret them.This survey focuses on data-driven analysis for quanti-tative understanding of invasion team sports behaviorssuch as basketball and football, and introduces twomain approaches for understanding such multi-agentbehaviors: (1) extracting easily interpretable featuresor rules from data and (2) generating and controllingbehaviors in visually-understandable ways. The firstapproach involves the visualization of learned repre-sentations and the extraction of mathematical struc-tures behind the behaviors. The second approach canbe used to test hypotheses by simulating and control-ling future and counterfactual behaviors. Lastly, thepotential practical applications of extracted rules, fea-tures, and generated behaviors are discussed. Theseapproaches can contribute to a better understandingof multi-agent behaviors in the real world.Keywords:
Human behavior, Machine learning, Dynam-ical systems, Sports, Interpretability
1. Introduction
The development of measurement technologies hasmade possible the measurement and analysis of the move-ments of various organisms. For example, they have en-abled an understanding of the behaviors of wild animalsand athletes from data. Specifically, recent advances insports-related measurement technologies have been re-viewed by many researchers such as in [1, 2, 3]. Based onthe advances, it is now possible to obtain a better under-standing of the principles of real-world biological multi-agent behaviors, which is a fundamental problem in var-ious scientific and engineering fields. The rules under-lying real-world biological multi-agent behaviors are of- ten largely unknown because the elements are not phys-ically connected. Mathematical models based on sim-ple rules are used to directly understand the multi-agentmovements. For example, models based on social forcesare widely applied, in which a force is assumed to be act-ing among individuals [4]. In a limited number of situa-tions, these models are also applied to more complicatedbehaviors such as sports [5, 6, 7]. However, modelingthe general multi-agent behaviors of living organisms inthe real world (e.g., team sports) can be mathematicallydifficult due to their inherently higher-order social inter-actions, cognition, and body dynamics [8]. Therefore, toobtain a better understanding of these behaviors, a data-driven and model-free (or equation-free) approach [9, 10]is needed.Data-driven modeling is a powerful approach such asfor extracting information and making a prediction us-ing complex real-world data. For example, learning mod-els with complex nonlinear structures such as neural net-works, are actively studied in the field of machine learn-ing. Although these nonlinear models are often effectivein terms of obtaining higher expressiveness and predic-tive performance, they are sometimes difficult to inter-pret. Hence, this study aims to bridge the gap betweenrule-based (or traditional sports sciences) and data-drivenapproaches, for which there is a trade-off between in-terpretability and expressiveness (or predictability). Thenext questions must be: what kind of nonlinear data-driven model will enable a better quantitative understand-ing? In a discussion of this issue regarding the rela-tionship between cognitive science and deep neural net-work models [11], the authors mentioned that such mod-els would have value if they could predict and explainphenomena, which could serve as a starting point for theestablishment of new theories. In the case of complexmulti-agent behaviors, existing rule-based models are toosimple. To obtain a better understanding, indirect tech-niques using nonlinear data-driven models are required:e.g., (i) extracting the mathematical structure behind themotions, (ii) visualizing the learned representations, and(iii) modeling the components and generating plausiblemotions. If this requirement can be satisfied, even the re-sults are based on a nonlinear data-driven model, it willbe possible to contribute to the understanding of complexmulti-agent behaviors.In this paper, data-driven analyses for team sports be-haviors are introduced, especially in invasion sports suchas basketball and football, which show complex interac-under review 1 eisuke Fujii tive behaviors. A range of related surveys or disserta-tions have addressed the spatio-temporal aspects of thisissue [12, 13, 14] with a focus on football [15], andhave discussed prediction approaches via machine learn-ing [16, 17, 18] including match outcome prediction, tac-tical decision making, player investments, fantasy sports,and injury prediction. The contribution of this paper is toreview data-driven analyses that interpret team sports be-haviors (e.g., based on the trajectory and action data of theplayers and ball as defined in Section 2), rather than sim-ply performing clustering, classification, and predictionvia black-box learning-based models. After the prelimi-nary explanation of terms in Section 2, examples of data-driven approaches to extract features and rules are intro-duced in Section 3, including the visualization of learnedrepresentations and extraction of mathematical structuresunderlying the behaviors. In Section 4, an approach fortesting hypotheses by simulating and controlling plausiblefuture behaviors by generating future and counterfactualbehaviors is introduced. Lastly, in Section 5, the poten-tial for the practical application of these estimated rules,features, and generated predictions is discussed.
Fig. 1.
An example of multi-agent trajectory data in bas-ketball (illustration from [19]). The colored triangles, graycircles, and the black circle represent the defenders, attack-ers, and ball, respectively.
Fig. 2.
An example of a player action sequence with a ballin soccer (illustration from [20]).
2. Preliminary
The term agent is used to denote a dynamic object ofinterest such as a player or the ball in team sports.
Asingle-agent trajectory P of length m is a sequence of m features P = ( p , p , . . . , p m ) , where p i ∈ R d is a fea-tures with d dimensions. For example, as a feature, the d -dimensional coordinate is a simple case. Multi-agenttrajectories P K (e.g., a team, both teams, or with theball, such as in Figure 1) with K agents comprise a se-quence of m features P K = ( p K , , p K , , . . . , p K , m ) , where p K , i = [ p i , , p i , , . . . p i , K ] ∈ R K × d . A sequence of rela-tions in a multi-agent system RRR K is defined as RRR K =( R , R , . . . , R m ) , where R i ∈ R K × K . R i ’s component R i , k , l represents the relation between agents k and l at each i , ascomputed by R i , k , l = h ( p i , k , p i , l ) (e.g., h is a distance func-tion [21, 9] or a Gaussian kernel [22, 23]). In the follow-ing, actions indicate discrete behaviors such as dribble,pass, and shot, as shown in Figure 2. The objective of thispaper is to present a method for obtaining a better under-standing of team sports behaviors , including continuoustrajectories and discrete actions.
3. Extracting features and rules from data
This learning-based approach is used to extract featuresand rules despite the availability of little prior knowl-edge. In this section, conventional rule-based approachesare firstly introduced, followed by unsupervised and su-pervised learning approaches, with particular regard totheir interpretation. The unsupervised and supervised ap-proaches comprise two of the three main categories of ma-chine learning (the third is reinforcement learning, whichis introduced in Section 4).
In conventional methods without learning-based ap-proaches, researchers in various fields have evaluated thecharacteristics of multi-agent behaviors based on their ex-perience and established theories. For example, basedon hypotheses, researchers have calculated the distancesand relative phases of two athletes (e.g., [24, 25, 8]), thespeeds of movements (e.g., [26]), the frequencies and an-gles of actions (e.g., shots [27] and passes [28, 29, 30]),and their representative values (e.g., average and max-imum values). Measurement systems with greater spa-tiotemporal resolution (e.g., motion capture systems andforce platforms) can analyze skillful maneuvers [31, 32]in terms of their cognition [33, 34], force [35], and torque[36]. After obtaining representative values, specific hy-potheses have been tested (e.g., [8, 37]) sometimes bystatistical analysis. For example, in order to quantifythe flexible teamwork of basketball defense (i.e., 5-vs-5),evaluation of the defensive cooperation against team at-tacks called screen-plays, which block the movements ofa defender, was performed [8]. The results showed thatthe defender flexibly changes the frequency of the four2 under review achine Learning based Analysis for Team Sports Behaviors roles (i.e., switching, overlapping, ignoring, global-help)according to the level of the emergency. This traditionalquantitative approach remains powerful, is applicable tosmall datasets, and is the easiest to interpret in a rangeof fields (e.g., a particular sport) because it allows for thedirect test of the hypothesis.Representative values have also been computed usingmore mathematically sophisticated approaches. Pioneer-ing work was conducted in which each player’s area ofcontrol in actual soccer games was evaluated as a Voronoidiagram [38]. Other studies, for example, have analyzedthe connection of passes based on network theory [39],the self-similarity hidden in a time series of the front po-sition of the team [40], and the breaking of spatiotemporalsymmetry using group theory in a 3-vs-1 ball possessiontask [41]. In a recent work [42], a probabilistic physics-based model was developed to quantify off-ball scoringopportunities. However, in order to represent coopera-tive/competitive interactions in a more detailed or prac-tical manner, more flexible modeling would be needed.A wide variety of data-driven methods such as using ma-chine learning have basically been developed to achieveclear objectives such as automatic feature extraction, clas-sification, and regression. In the following subsection,unsupervised and supervised learning, which are used inthe field of machine learning [43] are introduced, and ex-amples of researches using player position data in teamsports, which can be readily interpreted, are presented.
Unsupervised learning involves a type of machinelearning algorithm that acquires insight by inferring afunction for describing hidden structures from unlabeleddata. This is a powerful approach for knowledge dis-covery from data without the benefit of clear hypothe-ses. Typical unsupervised methods include dimensional-ity reduction and clustering. Dimensionality reduction isthe transformation of high-dimensional data into a mean-ingful representation of lower dimensionality. For ex-ample, principal component analysis (PCA), t-distributedstochastic neighbor embedding (t-SNE) [44] regardingshot type [45], non-negative matrix factorization (NMF)[46] or tensor decomposition [47] regarding the shot type,and topic modeling [48, 49] of the trajectory (i.e., one ofthe natural language processing algorithms), have beenused to summarize diverse interactive sports behaviorsinto lower-dimensional representations. However, someof these methods have assumed independence of sam-pling. That is, the extracted information does not reflectdynamical properties. Therefore, an extraction methodidentifying the coordinative structures based on dynam-ical properties from data is needed.A number of approaches are used to reduce the num-ber of dimensions while considering the time-series struc-tures. For example, image-based approaches trans-form trajectory data into images using neural networks(e.g., [50, 51]), including the self-organizing map (e.g.,[52, 53]). Another approach for extracting physically-interpretable dynamical information is a method called dynamic mode decomposition (DMD) [54, 55]. It can de-compose data into a small number of time dynamics (i.e.,frequency and growth rate) and their coefficients (i.e., ex-traction of dynamic properties). DMD is based on thespectral theory of the Koopman operator [56, 57]. Theo-retically, to compute DMD, the data must be rich enoughto approximate the eigenfunctions of the Koopman oper-ator. However, in basic DMD algorithms that naively usethe obtained data, the above assumption is not satisfiede.g., when the data dimension is too small to approximatethe eigenfunctions. Thus, there are several algorithmicvariants of DMDs to overcome this problem such as a for-mulation in reproducing kernel Hilbert spaces (RKHSs)[58], in a multitask framework [59], and using a neu-ral network [60]. Researchers have applied the DMD inRKHSs to multi-agent relation sequences (see Section 2in team sports [21, 9] and utilized the structure of an ad-jacency matrix series
RRR K (see Section 2) via tensor-traindecomposition [23]. This approach has the advantage ofenabling (i) the extraction of the mathematical structureand (ii) visualization of the learned expressions for theabove purposes of data-driven methods.Clustering involves grouping a set of objects such thatobjects in the same group (called a cluster) are more sim-ilar to each other than to those in other groups (clus-ters). There are many clustering algorithms based on var-ious cluster models, e.g., hierarchical clustering (basedon the connectivity or similarity between two trajecto-ries), centroid-based clustering (such as k-means), anddistribution-based clustering (such as Gaussian mixturemodels). For team sports data, researchers have usedhierarchical clustering [61, 62] based on similarity [63,64, 65] and distribution-based clustering using a Gaus-sian mixture model [66]. However, again, problems canoccur when using time-series data (for example, it is diffi-cult to naively compute a similarity when the data do nothave fixed time lengths). In that case, one approach is tospecifically design the similarity of time series to enablethe application of the conventional clustering method tostatic data.Hierarchical clustering requires appropriate distancemeasures. Among the several distance measures availablefor trajectories, the Fréchet distance [67] and dynamictime warping (DTW) [68] have been frequently used (e.g.,in basketball [63, 64] soccer [63] games). However, thesesimple approaches have high computational costs and aredifficult to apply to large-scale sports data. Therefore, re-searchers have developed a scalable method for comput-ing Fréchet distance by quickly performing a search ona tree data structure called trie [65]. Recently developedneural network approaches can also compute the similari-ties of a single-agent trajectory in scalable ways [69, 70],but these have not been applied to team sports multi-agenttrajectory datasets.Another problem is the computation of the distanceor similarity between multi-agent trajectories. A simplemethod for comparing agent-to-agent trajectories encoun-ters permutation problems among the players [64]. Onerule-based approach permutes the players nearest to theunder review 3 eisuke Fujii ball used such as in [9, 23]. A data-driven permutationmethod such as a linear assignment, known as the Hun-garian algorithm [71], has also been used for role assign-ment problems in Basketball [72, 73, 64, 74] and soccer[75, 76] (e.g., guard, forward, and center in basketball).Another approach to deal with the permutation prob-lem is calculating the similarity of multivariate nonlineardynamical systems using DMD [21, 9]. Since DMD is adimensionality reduction method like PCA, the extracteddynamical property is permutation-invariant. Moreover,this approach uses a kernel that reflects the dynamics viathe extraction of dynamical properties. A kernel called theKoopman spectral kernel can be regarded as a similaritybetween multivariate nonlinear dynamical systems, whichpermits the use of some clustering methods. However, ingeneral, since unsupervised learning methods do not useobjective variables (labeled data), it is sometimes difficultto validate them quantitatively. To evaluate them quan-titatively, combining them with the following supervisedlearning methods may be effective. Supervised learning is a machine learning task of infer-ring a function from supervised or labeled training data.When labeled data has discrete values such as the type ofplay, it is called classification, and when it has (relatively)continuous values such as position and score, it is referredto as regression. Here, classification problems of teamplays or regression problems for scoring probability areconsidered (other regression problems such as trajectoryprediction are described later). A simple approach is to in-put static features into classification or regression models.For example, score prediction in basketball [77, 78, 79],team identification in soccer [80], screen-play classifica-tion, [81, 82, 83], and prediction of who will obtain a bas-ketball in rebounding situations [84] using such as lineardiscriminant analysis (LDA), logistic regression, or sup-port vector machine (SVM) with the hand-crafted staticfeatures described in Section 3.1. In this process, thestatic features obtained from unsupervised learning (e.g.,[85]) can be input into classification or regression models.However, it is often necessary to reflect the time-seriesstructure also when supervised learning is applied to com-plex multi-agent behaviors. A simple approach is to usethe dynamic features obtained from unsupervised learn-ing, as described in Section 3. For example, by the use ofthe above DMD and computation of similarity, defensivetactics (defending the area or players) and offensive tac-tics (with or without cooperation) [23] can be classified.Another supervised learning method has also been usedto classify and predict the scoring probability [21, 9]. Thestrength of supervised learning is that the results can beclearly evaluated.More sophisticated approaches are end-to-end ap-proaches, which use the same model to extract featuresand perform predictions (i.e., classification or regression).For example, a neural network approach can be used toclassify offensive plays [50], team styles [86], and attack outcomes based on evaluating micro-actions [87]. How-ever, a neural network approach sometimes lacks inter-pretability. To obtain interpretable spatial representations,researchers have developed a number of approaches thatprovide both predictability and interpretability, such asusing matrix [88] and tensor [89, 90] factor models, andPoisson point process [91]. Other researchers have ap-plied a supervised pattern mining method to rugby eventdata [92], which can also be applied to trajectory data af-ter transforming the data into symbol sequences.The combinations of the predictability and inter-pretability are related to practical applications to actualsports games because coaches and players need informa-tion such as why the score was obtained and what charac-teristics are observed in the subsequent plays. To explainand understand multi-agent behaviors more quantitativelyor practically, it is necessary not only to improve pre-diction performance, but also to clarify their underlyingprinciples (e.g., identify the mathematical structure andprovide visualized representations that are interpretable).Meanwhile, if the purpose of an analysis is close to itspractical application, such as simulating and controllingbehaviors as discussed in the next Section 4, there maybe no problems in using even black-box learning-basedmodels.
4. Simulating and controlling behaviors
This approach enables verification of researchers’ hy-pothesis by modeling for future prediction or in situationsthat cannot be actually measured. In this section, con-ventional rule-based (or physics-based) approaches are in-troduced, followed by pattern-based (or data-driven) andplanning-based approaches, based on the categorizationof a human trajectory prediction survey [93]. Pattern-based methods approximate an arbitrary dynamics func-tion from training data to discover statistical behavioralpatterns. Planning-based methods explicitly address long-term movement goals of an agent and compute policies orpath hypotheses that enable the agent to reach those goals(often formulated as reinforcement learning).
Traditionally, rule-based (or physics-based) methodsenable researchers to determine and model the parame-ters of models (e.g., player position, speed, and interac-tion with other players). For example, the movementsof players in a 3-vs-1 soccer possession task was simplymodeled using three virtual social forces: spatial, avoid-ing, cooperative forces [5]. In actual soccer games, passprobabilities [6] and the future trajectories of players inseveral seconds [7] have been modeled using more com-plex rule-based approaches. These approaches have theadvantage of providing an understanding of simulated andcontrolled behaviors because the users set all of the pa-rameters. However, the adaptation of this approach to dif-ferent problems (e.g., from soccer to basketball) requiresadditional and costly human labor.4 under review achine Learning based Analysis for Team Sports Behaviors
Pattern-based or data-driven approaches learn dynam-ics from data using less human knowledge to solve theabove problem. In studies of team sports, there have beenmainly two goals in applying these approaches: simu-lating multi-agent trajectories over several seconds anda more long-term team outcome. To predict long-termoutcomes, if short-term behaviors are ignored, it is possi-ble to simulate behaviors until the end of the possession(or attack). Although this methodology mainly involvessupervised learning, which overlaps with the content inSection 3.3, these methods are used to simulate and eval-uate player behaviors, rather than extracting features andrules. In particular, researchers can use the reinforcementlearning framework to evaluate either a player’s actionand state, or the team state to achieve the goals describedin the following Section 4.3. In this subsection, modelingmethods of multi-agent trajectories are then introduced.4.2.1. Simulating multi-agent trajectoriesThe prediction of even just a few seconds of the multi-agent trajectories in team sports, e.g., basketball and soc-cer, is generally difficult. That is why it is one of thebenchmark problems in the field of machine learning[94, 75, 95, 76, 96]. Most methods have leveraged re-current neural networks (RNN) [94, 75, 97, 98] includingvariational RNNs [95, 76], although some have utilizedgenerative adversarial networks (e.g., [99, 100]) and vari-ational autoencoders [101] without RNNs. Most of thesemethods were simply formulated as a trajectory predictionproblem, whereas a few studies formulated it as an imita-tion learning problem (e.g., [75, 19], which is one of rein-forcement learning framework utilizing demonstration ofexperts (i.e., data).Most of these methods assume full observation toachieve long-term prediction in a centralized manner(e.g., [95, 76]). In such a case, an important latent fac-tor, e.g., whose information is utilized by each agent,is not interpretable. Methods for learning attention-based observation of agents have been proposed for multi-agent in virtual environments and in real-world sys-tems [102, 103, 19]. Other approaches such as rela-tional (e.g., [104, 105]) or a physically-interepretable ap-proaches [22, 23] can learn interpretable representationsof interactions. Rigorously, decentralized modeling [19]is needed to enable computation of each agent observation(or contribution). Meanwhile, recent graph neural net-work approaches can learn permutation-equivariant fea-tures [104, 76, 106, 105], which solve the permutationproblem described in Section 3.2.Another important approach is the tactical evaluationof a predicted trajectory. For example, trajectory predic-tion reflecting defensive evaluations in soccer [107] andtrajectory computation optimizing defensive evaluationsin basketball [14]. Qualitatively, the evaluation of coun-terfactual prediction (i.e., predicting "what if" situations)can be used to validate the models [76, 19] based on theuser’s knowledge, whereas there is no ground truth in a counterfactual situation.Although it is generally difficult to extract mathemati-cal structures with such an approach that prioritizes pre-dictive performance and performs a nonlinear transforma-tion, there are methods that make them compatible such asin [108] with applications other than sports. Such meth-ods can be useful for explicit modeling (e.g., observationmodel) of the nonlinear model when the phenomenon canbe sufficiently explained or used as a starting point for var-ious theories [11] as mentioned in the Introduction. Theseapproaches enable realistic and visually-understandablesimulations (e.g., average athlete movements and the re-sponse to unobserved movements). Potential practical ap-plications are presented in the following Section 5.
Planning-based methods explicitly address the long-term movement goals of agents and compute policies orpath hypotheses that enable the agent to reach those goals.According to [93], planning-based approaches are clas-sified into two categories: inverse and forward planningmethods. Inverse planning methods estimate the actionmodel or reward function from observed data using sta-tistical learning techniques. In other words, this approachutilizes a reinforcement learning framework in physicalspaces (or in real-world data). Although it sometimesoverlaps with supervised learning in Section 3.3 and im-itation learning in Section 4.2, the methods introducedhere are used to evaluate actions and states of a playeror a team to achieve their goals, rather than to extract fea-tures and rules or predict trajectories. Forward planningmethods make an explicit assumption regarding the opti-mal criteria of an agent’s movements, using a pre-definedreward function (e.g., a score in team sports). These twoapproaches are described in this subsection.4.3.1. Inverse approach using real-world dataThe inverse planning approach uses statistical learningtechniques to estimate an action model or reward func-tion from observed data. Here this idea is extended toconsider and value players’ actions and the team’s states.For example, with respect to shooting, valuing player’sactions by estimating the scoring and conceding probabil-ity (VAEP) [20] and estimating a state-action value func-tion (Q-function) using an RNN [109, 110], which madeinterpretable using a linear model tree [111]. To evaluatethe shooting action of players, researchers have investi-gated allocative efficiency in shot [112], the expected pos-session value [77, 113, 114], and the value of the space[115, 116] by extending a Voronoi diagram [38]. Regard-ing passing actions, similarly, researchers have used mod-eling and valuing of a pass [117, 118, 119], pass-receiving[120], the defender’s pass-interception [121]. In teamplays, deep reinforcement learning to estimate the qual-ity of the defensive actions was used in ball-screen de-fense in basketball [122]. Another approach is the qualita-tive evaluation of counterfactual predictions as describedunder review 5 eisuke Fujii above. For example, researchers have modeled the tran-sition probabilities and shot policy tensors and have sim-ulated seasons under alternative shot policies of interest[123].4.3.2. Forward approach in virtual spacesThe forward planning approach involves the develop-ment of algorithms for the purpose of winning a com-petition involving humans or machines in virtual space(e.g., video games). To develop methods both in physicaland virtual spaces, RoboCup (the Robot World Cup Ini-tiative) involves attempts by robot teams to actually playa soccer game [124]. Research has been conducted onvirtual games such as puzzles and shooters, and recentlyan open-source simulator for soccer games was published[125]. Some researchers used a 3-vs-3 basketball simu-lator [126], which is not currently open-source. In thesestudies, using reinforcement learning, the performancesare expected to defeat humans in some cases (such as mas-tering the game of Go [127]). It is also possible to learnsimilar behaviors from measurement data in sports games(e.g., using imitation learning frameworks as mentionedabove). However, a few studies have combined inverseand forward planning-based frameworks. For example,reinforcement learning could generate the optimal defen-sive team trajectory with the reward of preventing oppo-nent scores after the imitation learning [64]. An approachto bridge this gap is an important issue for future research.
5. Practical applications and future directions
There are a number of possible practical applications ofextracted rules, features, and generated behaviors. First,if play classification and score prediction become possi-ble based on the extracted features and rules of multi-agent behaviors, the most directly useful application is thedecrease in the workload of those who manually classi-fied and evaluated plays by watching videos. However, itwould be sometimes difficult to define specific plays thatthe user wishes to classify, whereas other plays can beeasily defined (e.g., offensive and defensive tactics [23]in basketball). In such a case, it may be possible to col-lect similar plays in the form of a recommendation systembased on unsupervised learning (in Section 3.2), such asin an analogous way of a search on a web page.Regarding the short-term future prediction discussed inSection 4, these can visually present e.g., how will a cer-tain move work for a player in the same situation as agood player, and how will the team in the next game re-spond to our team. In long-term prediction, predictingthe game situations and results of the opponent team inthe next game would be useful for tactical planning pur-poses. Although there are gaps between the resolutionof practical application and research on tactical planningin invasive sports (e.g., formations and styles in soccer[128] and specific cooperative plays and defense styles inbasketball [23]), other team sports such as baseball have fewer such gaps [129] because most of their actions can beevaluated discretely. Since individual results can be moreeasily predicted in invasive team sports (especially thosenear the ball), many studies in recent years have evaluatedthe behaviors of professional athletes (e.g., [20, 130]).Three possible future issues can be considered. Oneis the high cost of using location information, which cur-rently limits its usage to professional sports. This prob-lem is being researched with respect to both software andhardware, and we expect that it may become easier to ob-tain and more accurate in the future, even for estimatingjoint positions [131]. With greater spatiotemporal resolu-tion, skillful maneuver in terms of their cognition, force,and torque can be analyzed as described in Section 3.1.The second is that higher (almost perfect) performanceis often required for practical use. However, it may bemore constructive to consider whether the results obtainedby these approaches are better (less expensive with fewermistakes) than those obtained by humans.
6. Conclusions
This survey focused on data-driven analyses that canbe used to obtain a quantitative understanding of invasionteam sports behaviors. Two approaches for understand-ing these multi-agent behaviors were introduced: (1) theextraction of features or rules from data in interpretableways and (2) the generation and control of behaviors invisually-understandable ways. Lastly, the potential prac-tical applications of extracted rules, features, and gener-ated behaviors were discussed. The development of theseapproaches would contribute to a better understanding ofmulti-agent behaviors in the real world.
Acknowledgements
I would like to thank Atom Scott, Masaki Onishi, and RoryBunker for their valuable comments on this work. This workwas supported by JSPS KAKENHI (Grant Numbers 19H04941,20H04075, 20H04087) and JST Presto (Grant Number JP-MJPR20CA).
References: [1] Nur Azmina Rahmad, Muhammad Amir As’Ari, Nurul FathiahGhazali, Norazman Shahar, and Nur Anis Jasmin Sufri. A surveyof video based action recognition in sports.
Indonesian Journalof Electrical Engineering and Computer Science , 11(3):987–993,2018.[2] Paresh R Kamble, Avinash G Keskar, and Kishor M Bhurchandi.Ball tracking in sports: a survey.
Artificial Intelligence Review ,52(3):1655–1705, 2019.[3] Markel Rico-González, José Pino-Ortega, Fabio Y Nakamura, Fe-lipe Arruda Moura, Daniel Rojas-Valverde, and Asier Los Arcos.Past, present, and future of the technological tracking methods toassess tactical variables in team sports: A systematic review.
Pro-ceedings of the Institution of Mechanical Engineers, Part P: Journalof Sports Engineering and Technology , pages 281–290, 2020.[4] Dirk Helbing and Peter Molnar. Social force model for pedestriandynamics.
Physical Review E , 51(5):4282, 1995.[5] Keiko Yokoyama, Hiroyuki Shima, Keisuke Fujii, NoriyukiTabuchi, and Yuji Yamamoto. Social forces for team coordinationin ball possession game.
Physical Review E , 97(2):022410, 2018.[6] William Spearman, Austin Basye, Greg Dick, Ryan Hotovy, and achine Learning based Analysis for Team Sports Behaviors
Paul Pop. Physics-based modeling of pass probabilities in soccer.In
Proceeding of the MIT Sloan Sports Analytics Conference , 2017.[7] Francisco Peralta Alguacil, Pablo Piñones Arce, David Sumpter,and Javier Fernandez. Seeing in to the future: using self-propelledparticle models to aid player decision-making in soccer. In
Pro-ceedings of the MIT Sloan Sports Analytics Conference , 2020.[8] Keisuke Fujii, Keiko Yokoyama, Takeshi Koyama, AkiraRikukawa, Hiroshi Yamada, and Yuji Yamamoto. Resilient help toswitch and overlap hierarchical subsystems in a small human group.
Scientific Reports , 6, 2016.[9] Keisuke Fujii, Takeshi Kawasaki, Yuki Inaba, and YoshinobuKawahara. Prediction and classification in equation-free collectivemotion dynamics.
PLoS Computational Biology , 14(11):e1006545,2018.[10] Keisuke Fujii, Naoya Takeishi, Benio Kibushi, Motoki Kouzaki,and Yoshinobu Kawahara. Data-driven spectral analysis for coordi-native structures in periodic human locomotion.
Scientific Reports ,9(1):1–14, 2019.[11] Radoslaw M Cichy and Daniel Kaiser. Deep neural networks as sci-entific models.
Trends in Cognitive Sciences , 23(4):305–317, 2019.[12] Joachim Gudmundsson and Michael Horton. Spatio-temporal anal-ysis of team sports.
ACM Computing Surveys (CSUR) , 50(2):1–34,2017.[13] Michael Horton.
Algorithms for the Analysis of Spatio-TemporalData from Team Sports . PhD thesis, University of Sydney, 2018.[14] Long Sha.
Representing and predicting multi-agent data in adver-sarial team sports . PhD thesis, Queensland University of Technol-ogy, 2018.[15] Karl Tuyls, Shayegan Omidshafiei, Paul Muller, Zhe Wang, JeromeConnor, Daniel Hennes, Ian Graham, William Spearman, Tim Was-kett, Dafydd Steele, et al. Game plan: What AI can do for football,and what football can do for AI. arXiv preprint arXiv:2011.09192 ,2020.[16] Ryan Beal, Timothy J Norman, and Sarvapali D Ramchurn. Arti-ficial intelligence for team sports: a survey.
The Knowledge Engi-neering Review , 34:e28, 2019.[17] Rory P Bunker and Fadi Thabtah. A machine learning frameworkfor sport result prediction.
Applied Computing and Informatics ,15(1):27–33, 2019.[18] Milad Keshtkar Langaroudi and Mohammadreza Yamaghani.Sports result prediction based on machine learning and computa-tional intelligence approaches: A survey.
Journal of Advances inComputer Engineering and Technology , 5(1):27–36, 2019.[19] Keisuke Fujii, Naoya Takeishi, Yoshinobu Kawahara, and KazuyaTakeda. Policy learning with partial observation and me-chanical constraints for multi-person modeling. arXiv preprintarXiv:2007.03155 , 2020.[20] Tom Decroos, Lotte Bransen, Jan Van Haaren, and Jesse Davis. Ac-tions speak louder than goals: Valuing player actions in soccer. In
Proceedings of the 25th ACM SIGKDD International Conferenceon Knowledge Discovery & Data Mining , pages 1851–1861, 2019.[21] Keisuke Fujii, Yuki Inaba, and Yoshinobu Kawahara. Koopmanspectral kernels for comparing complex dynamics: Application tomultiagent sport plays. In
European Conference on Machine Learn-ing and Knowledge Discovery in Databases (ECML-PKDD’17) ,pages 127–139. Springer, 2017.[22] Keisuke Fujii and Yoshinobu Kawahara. Dynamic mode decom-position in vector-valued reproducing kernel hilbert spaces for ex-tracting dynamical structure among observables.
Neural Networks ,117:94–103, 2019.[23] Keisuke Fujii, Naoya Takeishi, Motokazu Hojo, Yuki Inaba, andYoshinobu Kawahara. Physically-interpretable classification of net-work dynamics for complex collective motions.
Scientific Reports ,10(3005), 2020.[24] Jérôme Bourbousson, Carole Sève, and Tim McGarry. Space–timecoordination dynamics in basketball: Part 2. the interaction betweenthe two teams.
Journal of Sports Sciences , 28(3):349–358, 2010.[25] B Travassos, D Araújo, R Duarte, and T McGarry. Spatiotem-poral coordination behaviors in futsal (indoor football) are guidedby informational game constraints.
Human Movement Science ,31(4):932–945, 2012.[26] Jaime Sampaio, Tim McGarry, Julio Calleja-González, SergioJiménez Sáiz, Xavi Schelling i del Alcázar, and Mindaugas Bal-ciunas. Exploring game performance in the national basketball as-sociation using player tracking data.
PLoS One , 10(7):e0132894,2015.[27] Kirk Goldsberry. Courtvision: New visual and spatial analytics forthe nba. In , volume 9,pages 12–15, 2012.[28] Vanda Correia, Duarte Araujo, Cathy Craig, and Pedro Passos.Prospective information for pass decisional behavior in rugbyunion.
Human Movement Science , 30(5):984–997, 2011. [29] Luís Vilar, Duarte Araújo, Keith Davids, Vanda Correia, and Pe-dro Tiago Esteves. Spatial-temporal constraints on decision-makingduring shooting performance in the team sport of futsal.
Journal ofSports Sciences , 31(8):840–846, 2013.[30] Keisuke Fujii, Yujiro Yoshihara, Yukiko Matsumoto, KeimaTose, Hideaki Takeuchi, Masanori Isobe, Hiroto Mizuta, DaisukeManiwa, Takehiko Okamura, Toshiya Murai, et al. Cognition andinterpersonal coordination of patients with schizophrenia who havesports habits.
PLoS One , 15(11):e0241863, 2020.[31] Keisuke Fujii, Daichi Yamashita, Shinsuke Yoshioka, Tadao Isaka,and Motoki Kouzaki. Strategies for defending a dribbler: cate-gorisation of three defensive patterns in 1-on-1 basketball.
SportsBiomechanics , 13(3):204–214, 2014.[32] Keisuke Fujii, Tadao Isaka, Motoki Kouzaki, and Yuji Yamamoto.Mutual and asynchronous anticipation and action in sports as glob-ally competitive and locally coordinative dynamics.
Scientific Re-ports , 5, 2015.[33] Sebastien Brault, Benoit Bideau, Richard Kulpa, and Cathy MCraig. Detecting deception in movement: the case of the side-stepin rugby.
PLoS One , 7(6):e37494, 2012.[34] Keisuke Fujii, Masahiro Shinya, Daichi Yamashita, MotokiKouzaki, and Shingo Oda. Anticipation by basketball defenders:An explanation based on the three-dimensional inverted pendulummodel.
European Journal of Sport Science , 14(6):538–546, 2014.[35] Keisuke Fujii, Shinsuke Yoshioka, Tadao Isaka, and MotokiKouzaki. The preparatory state of ground reaction forces in de-fending against a dribbler in a basketball 1-on-1 dribble subphase.
Sports Biomechanics , 14(1):28–44, 2015.[36] Keisuke Fujii, Daichi Yamashita, Tetsuya Kimura, Tadao Isaka, andMotoki Kouzaki. Preparatory body state before reacting to an op-ponent: Short-term joint torque fluctuation in real-time competitivesports.
PLoS One , 10(5):e0128571, 2015.[37] Paul Power, Jennifer Hobbs, Hector Ruiz, Xinyu Wei, and PatrickLucey. Mythbusting set-pieces in soccer.
Proceedings of the MITSloan Sports Analytics Conference , 2018.[38] Tsuyoshi Taki and Jun-Ichi Hasegawa. Visualization of dominantregion in team games and its application to teamwork analysis.In
Proceedings Computer Graphics International , pages 227–235,2000.[39] Yuji Yamamoto and Keiko Yokoyama. Common and unique net-work dynamics in football games.
PLoS One , 6(12):e29638, 2011.[40] Akifumi Kijima, Keiko Yokoyama, Hiroyuki Shima, and Yuji Ya-mamoto. Emergence of self-similarity in football dynamics.
TheEuropean Physical Journal B , 87(2):41, 2014.[41] Keiko Yokoyama and Yuji Yamamoto. Three people can synchro-nize as coupled oscillators during sports activities.
PLoS ComputBiol , 7(10):e1002181, 2011.[42] William Spearman. Beyond expected goals. In
Proceedings of theMIT Sloan Sports Analytics Conference , pages 1–17, 2018.[43] Christopher M Bishop.
Pattern recognition and machine learning .springer, 2006.[44] Laurens van der Maaten and Geoffrey Hinton. Visualizing data us-ing t-SNE.
Journal of Machine Learning Research , 9:2579–2605,2008.[45] Rachel Marty. High-resolution shot capture reveals systematic bi-ases and an improved method for shooter evalutation. In
Proceed-ings of the 2018 MIT Sloan Sports Analytics Conference , 2018.[46] Andrew Miller, Luke Bornn, Ryan Adams, and Kirk Goldsberry.Factorized point process intensities: A spatial analysis of profes-sional basketball. In
International Conference on Machine Learn-ing , pages 235–243, 2014.[47] Evangelos Papalexakis and Konstantinos Pelechrinis. thoops: Amulti-aspect analytical framework for spatio-temporal basketballdata. In
Proceedings of the 27th ACM International Conference onInformation and Knowledge Management , pages 2223–2232, 2018.[48] Qing Wang, Hengshu Zhu, Wei Hu, Zhiyong Shen, and Yuan Yao.Discerning tactical patterns for professional soccer teams: an en-hanced topic model with applications. In
Proceedings of the 21thACM SIGKDD International Conference on Knowledge Discoveryand Data Mining , pages 2197–2206, 2015.[49] Andrew C Miller and Luke Bornn. Possession sketches: MappingNBA strategies. In
Proceedings of the MIT Sloan Sports AnalyticsConference , 2017.[50] Kuan-Chieh Wang and Richard Zemel. Classifying nba offensiveplays using neural networks. In
Proceedings of the MIT SloanSports Analytics Conference , 2016.[51] Akhil Nistala.
Using deep learning to understand patterns of playermovement in basketball . PhD thesis, Massachusetts Institute ofTechnology, 2018.[52] Andreas Grunz, Daniel Memmert, and Jürgen Perl. Tactical patternrecognition in soccer games by means of special self-organizingmaps.
Human Movement Science , 31(2):334–343, 2012. under review 7 eisuke Fujii [53] Matthias Kempe, Andreas Grunz, and Daniel Memmert. Detectingtactical patterns in basketball: comparison of merge self-organisingmaps and dynamic controlled neural networks.
European Journalof Sport Science , 15(4):249–255, 2015.[54] Clarence W Rowley, Igor Mezi´c, Shervin Bagheri, Philipp Schlat-ter, and Dan S Henningson. Spectral analysis of nonlinear flows.
Journal of Fluid Mechanics , 641:115–127, 2009.[55] Peter J Schmid. Dynamic mode decomposition of numerical andexperimental data.
Journal of Fluid Mechanics , 656:5–28, 2010.[56] Bernard O Koopman. Hamiltonian systems and transformation inhilbert space.
Proceedings of the National Academy of Sciences ,17(5):315–318, 1931.[57] Igor Mezi´c. Spectral properties of dynamical systems, model re-duction and decompositions.
Nonlinear Dynamics , 41(1):309–325,2005.[58] Yoshinobu Kawahara. Dynamic mode decomposition with repro-ducing kernels for koopman spectral analysis. In
Advances in Neu-ral Information Processing Systems 29 , pages 911–919, 2016.[59] Keisuke Fujii and Yoshinobu Kawahara. Supervised dynamic modedecomposition via multitask learning.
Pattern Recognition Letters ,122(1):7–13, 2019.[60] Naoya Takeishi, Yoshinobu Kawahara, and Takehisa Yairi. Learn-ing koopman invariant subspaces for dynamic mode decomposition.In
Advances in Neural Information Processing Systems 30 , pages1130–1140, 2017.[61] Jennifer Hobbs, Paul Power, and Long Sha. Quantifying the valueof transitions in soccer via spatiotemporal trajectory clustering. In
Proceedings of the MIT Sloan Sports Analytics Conference , 2018.[62] Jennifer Hobbs, Matthew Holbrook, Nathan Frank, Long Sha, andPatrick Lucey. Improved structural discovery and representationlearning of multi-agent data. arXiv preprint arXiv:1912.13107 ,2019.[63] Tom Decroos, Jan Van Haaren, and Jesse Davis. Automatic dis-covery of tactics in spatio-temporal soccer match data. In
ACMSIGKDD International Conference on Knowledge Discovery &Data Mining , pages 223–232, 2018.[64] Long Sha, Patrick Lucey, Yisong Yue, Peter Carr, Charlie Rohlf,and Iain Matthews. Chalkboarding: A new spatiotemporal queryparadigm for sports play retrieval. In
International Conference onIntelligent User Interfaces , pages 336–347, 2016.[65] Shunsuke Kanda, Koh Takeuchi, Keisuke Fujii, and Yasuo Tabei.Succinct trit-array trie for scalable trajectory similarity search. arXiv preprint arXiv:2005.10917 , 2020.[66] Matej Perše, Matej Kristan, Stanislav Kovaˇciˇc, Goran Vuˇckoviˇc,and Janez Perš. A trajectory-based analysis of coordinated teamactivity in a basketball game.
Computer Vision and Image Under-standing , 113(5):612–621, 2009.[67] Helmut Alt and Michael Godau. Computing the Fréchet distancebetween two polygonal curves.
International Journal of Computa-tional Geometry & Applications , 5(01n02):75–91, 1995.[68] Donald J. Berndt and James Clifford. Using Dynamic Time Warp-ing to Find Patterns in Time Series. In
Proceedings of AAAI work-shop , 1994.[69] Di Yao, Chao Zhang, Zhihua Zhu, Jianhui Huang, and Jingping Bi.Trajectory clustering via deep representation learning. In , pages3880–3887. IEEE, 2017.[70] Xiucheng Li, Kaiqi Zhao, Gao Cong, Christian S Jensen, and WeiWei. Deep representation learning for trajectory similarity compu-tation. In , pages 617–628. IEEE, 2018.[71] Christos H Papadimitriou and Kenneth Steiglitz.
Combinatorial op-timization , volume 24. Prentice Hall Englewood Cliffs, 1982.[72] Patrick Lucey, Alina Bialkowski, Peter Carr, Stuart Morgan, IainMatthews, and Yaser Sheikh. Representing and discovering adver-sarial team behaviors using player roles. In
Proceedings of the IEEEConference on Computer Vision and Pattern Recognition , pages2706–2713, 2013.[73] Patrick Lucey, Alina Bialkowski, Peter Carr, Yisong Yue, and IainMatthews. How to get an open shot: Analyzing team movementin basketball using tracking data. In
Proceedings of the MIT SloanSports Analytics Conference , 2014.[74] Long Sha, Patrick Lucey, Stephan Zheng, Taehwan Kim, YisongYue, and Sridha Sridharan. Fine-grained retrieval of sportsplays using tree-based alignment of trajectories. arXiv preprintarXiv:1710.02255 , 2017.[75] Hoang M Le, Yisong Yue, Peter Carr, and Patrick Lucey. Coordi-nated multi-agent imitation learning. In
Proceedings of the 34th In-ternational Conference on Machine Learning 70 , pages 1995–2003,2017.[76] Raymond A Yeh, Alexander G Schwing, Jonathan Huang, andKevin Murphy. Diverse generation for multi-agent sports games.In
Proceedings of the IEEE Conference on Computer Vision andPattern Recognition , pages 4610–4619, 2019. [77] Dan Cervone, Alexander D’Amour, Luke Bornn, and Kirk Golds-berry. Pointwise: Predicting points and valuing decisions in realtime with nba optical tracking data. In
Proceedings of the MIT SloanSports Analytics Conference , 2014.[78] Yu-Han Chang, R. Maheswaran, Sheldon J. J. Kwok, Tal Levy,A. Wexler, and K. Squire. Quantifying shot quality in the nba. In
Proceedings of the MIT Sloan Sports Analytics Conference , 2014.[79] Patrick Lucey, Alina Bialkowski, Mathew Monfort, Peter Carr, andIain Matthews. quality vs quantity: Improved shot prediction insoccer using strategic features from spatiotemporal data. In
Pro-ceedings of the MIT Sloan Sports Analytics Conference , 2014.[80] Patrick Lucey, Dean Oliver, Peter Carr, Joe Roth, and IainMatthews. Assessing team strategy using spatiotemporal data. In
Proceedings of the 19th ACM SIGKDD International Conference onKnowledge Discovery and Data Mining , pages 1366–1374, 2013.[81] Armand McQueen, Jenna Wiens, and John Guttag. Automaticallyrecognizing on-ball screens. In
Proceedings of the MIT Sloan SportsAnalytics Conference , 2014.[82] Avery McIntyre, Joel Brooks, John Guttag, and Jenna Wiens. Rec-ognizing and analyzing ball screen defense in the NBA. In
Proceed-ings of the MIT Sloan Sports Analytics Conference , pages 11–12,2016.[83] Motokazu Hojo, Keisuke Fujii, Yuki Inaba, Yoichi Motoyasu, andYoshinobu Kawahara. Automatically recognizing strategic cooper-ative behaviors in various situations of a team sport.
PLoS One ,13(12):e0209247, 2018.[84] Motokazu Hojo, Keisuke Fujii, and Yoshinobu Kawahara. Analysisof factors predicting who obtains a ball in basketball reboundingsituations.
International Journal of Performance Analysis in Sport ,pages 1–14, 2019.[85] Alina Bialkowski, Patrick Lucey, Peter Carr, Yisong Yue, SridhaSridharan, and Iain Matthews. Identifying team style in soccer us-ing formations learned from spatiotemporal tracking data. In , pages9–14. IEEE, 2014.[86] Nazanin Mehrasa, Yatao Zhong, Frederick Tung, Luke Bornn, andGreg Mori. Deep learning of player trajectory representations forteam activity analysis. In
Proceedings of the MIT Sloan Sports An-alytics Conference , 2018.[87] Anthony Sicilia, Konstantinos Pelechrinis, and Kirk Goldsberry.Deephoops: Evaluating micro-actions in basketball using deep fea-ture representations of spatio-temporal data. In
Proceedings of the25th ACM SIGKDD International Conference on Knowledge Dis-covery & Data Mining , pages 2096–2104, 2019.[88] Yisong Yue, Patrick Lucey, Peter Carr, Alina Bialkowski, and IainMatthews. Learning fine-grained spatial models for dynamic sportsplay prediction. In , pages 670–679. IEEE, 2014.[89] Stephan Zheng, Rose Yu, and Yisong Yue. Multi-resolutiontensor learning for large-scale spatial data. arXiv preprintarXiv:1802.06825 , 2018.[90] Jung Yeon Park, Kenneth Theo Carr, Stephan Zhang, Yisong Yue,and Rose Yu. Multiresolution tensor learning for efficient and inter-pretable spatial analysis. arXiv preprint arXiv:2002.05578 , 2020.[91] Jacob Mortensen and Luke Bornn. From markov models to poissonpoint processes: Modeling movement in the nba. In
Proceedings ofthe 2019 MIT Sloan Sports Analytics Conference , 2019.[92] Rory Bunker, Keisuke Fujii, Hiroyuki Hanada, and Ichiro Takeuchi.Supervised sequential pattern mining of event sequences in sport toidentify important patterns of play: an application to rugby union. arXiv preprint arXiv:2010.15377 , 2020.[93] Andrey Rudenko, Luigi Palmieri, Michael Herman, Kris M Kitani,Dariu M Gavrila, and Kai O Arras. Human motion trajectory pre-diction: A survey.
The International Journal of Robotics Research ,39(8):895–935, 2020.[94] Stephan Zheng, Yisong Yue, and Jennifer Hobbs. Generating long-term trajectories using deep hierarchical networks. In
Advancesin Neural Information Processing Systems 29 , pages 1543–1551,2016.[95] Eric Zhan, Stephan Zheng, Yisong Yue, Long Sha, and PatrickLucey. Generating multi-agent trajectories using programmaticweak supervision. In
International Conference on Learning Rep-resentations , 2019.[96] Yukai Liu, Rose Yu, Stephan Zheng, Eric Zhan, and Yisong Yue.Naomi: Non-autoregressive multiresolution sequence imputation.In
Advances in Neural Information Processing Systems 32 , pages11236–11246, 2019.[97] Thomas Seidl, Aditya Cherukumudi, Andrew Hartnett, Peter Carr,and Patrick Lucey. Bhostgusters: Realtime interactive play sketch-ing with synthesized nba defenses. In
Proceedings of the MIT SloanSports Analytics Conference , 2018. achine Learning based Analysis for Team Sports Behaviors [98] Boris Ivanovic, Edward Schmerling, Karen Leung, and MarcoPavone. Generative modeling of multimodal multi-human behavior.In , pages 3088–3095. IEEE, 2018.[99] Chieh-Yu Chen, Wenze Lai, Hsin-Ying Hsieh, Wen-Hao Zheng, Yu-Shuen Wang, and Jung-Hong Chuang. Generating defensive playsin basketball games. In
Proceedings of the 26th ACM InternationalConference on Multimedia , pages 1580–1588, 2018.[100] Hsin-Ying Hsieh, Chieh-Yu Chen, Yu-Shuen Wang, and Jung-HongChuang. Basketballgan: Generating basketball play simulationthrough sketching. In
Proceedings of the 27th ACM InternationalConference on Multimedia , pages 720–728, 2019.[101] Panna Felsen, Patrick Lucey, and Sujoy Ganguly. Where will theygo? predicting fine-grained adversarial multi-agent motion usingconditional variational autoencoders. In
Proceedings of the Eu-ropean Conference on Computer Vision (ECCV) , pages 732–747,2018.[102] Yedid Hoshen. Vain: Attentional multi-agent predictive modeling.In
Advances in Neural Information Processing Systems 30 , pages2701–2711, 2017.[103] Li Guangyu, Jiang Bo, Zhu Hao, Che Zhengping, and Liu Yan.Generative attention networks for multi-agent behavioral modeling.In
Thirty-Fourth AAAI Conference on Artificial Intelligence , 2020.[104] Thomas Kipf, Ethan Fetaya, Kuan-Chieh Wang, Max Welling, andRichard Zemel. Neural relational inference for interacting systems.In
International Conference on Machine Learning , pages 2688–2697, 2018.[105] Colin Graber and Alexander G. Schwing. Dynamic neural rela-tional inference. In
Proceedings of the IEEE/CVF Conference onComputer Vision and Pattern Recognition (CVPR) , June 2020.[106] Chen Sun, Per Karlsson, Jiajun Wu, Joshua B Tenenbaum, andKevin Murphy. Predicting the present and future states of multi-agent systems from partially-observed visual data. In
InternationalConference on Learning Representations , 2019.[107] Masakiyo Teranishi, Keisuke Fujii, and Kazuya Takeda. Trajectoryprediction with imitation learning reflecting defensive evaluation inteam sports. In
IEEE 9th Global Conference on Consumer Elec-tronics (GCCE 2020) , 2020.[108] Marco Fraccaro, Simon Kamronn, Ulrich Paquet, and Ole Winther.A disentangled recognition and nonlinear dynamics model for un-supervised learning. In
Advances in Neural Information ProcessingSystems 30 , pages 3601–3610, 2017.[109] Guiliang Liu and Oliver Schulte. Deep reinforcement learning inice hockey for context-aware player evaluation. In
Proceedings ofthe 27th International Joint Conference on Artificial Intelligence ,pages 3442–3448, 2018.[110] Guiliang Liu, Yudong Luo, Oliver Schulte, and Tarak Kharrat.Deep soccer analytics: learning an action-value function for eval-uating soccer players.
Data Mining and Knowledge Discovery ,34(5):1531–1559, 2020.[111] Xiangyu Sun, Jack Davis, Oliver Schulte, and Guiliang Liu. Crack-ing the black box: Distilling deep sports analytics. In
Proceedingsof the 26th ACM SIGKDD International Conference on KnowledgeDiscovery & Data Mining , 2020.[112] Nathan Sandholtz, Jacob Mortensen, and Luke Bornn. Chuckers:Measuring lineup shot distribution optimality using spatial alloca-tive efficiency models. In
Proceedings of the MIT Sloan SportsAnalytics Conference , 2019.[113] Daniel Cervone, Alex D’Amour, Luke Bornn, and Kirk Goldsberry.A multiresolution stochastic process model for predicting basketballpossession outcomes.
Journal of the American Statistical Associa-tion , 111(514):585–599, 2016.[114] Javier Fernández, Luke Bornn, and Dan Cervone. Decomposingthe immeasurable sport: A deep learning expected possession valueframework for soccer. In
Proceedings of the MIT Sloan Sports An-alytics Conference , 2019.[115] Dan Cervone, Luke Bornn, and Kirk Goldsberry. NBA court realty.In
Proceedings of the MIT Sloan Sports Analytics Conference , 2016.[116] Javier Fernandez and Luke Bornn. Wide open spaces: A statisticaltechnique for measuring space creation in professional soccer. In
Proceedings of the MIT Sloan Sports Analytics Conference , 2018.[117] Paul Power, Hector Ruiz, Xinyu Wei, and Patrick Lucey. Not allpasses are created equal: Objectively measuring the risk and rewardof passes in soccer from tracking data. In
Proceedings of the 23rdACM SIGKDD International Conference on Knowledge Discoveryand Data Mining , pages 1605–1613, 2017.[118] Floris R Goes, Matthias Kempe, Laurentius A Meerhoff, andKoen APM Lemmink. Not every pass can be an assist: a data-driven model to measure pass effectiveness in professional soccermatches.
Big Data , 7(1):57–70, 2019.[119] Lotte Bransen, Jan Van Haaren, and Michel van de Velden. Measur-ing soccer players’ contributions to chance creation by valuing theirpasses.
Journal of Quantitative Analysis in Sports , 15(2):97–116,2019. [120] Sergio Llana, Pau Madrero, Javier Fernández, and FC Barcelona.The right place at the right time: Advanced off-ball metrics for ex-ploiting an opponent’s spatial weaknesses in soccer. In
Proceedingsof the MIT Sloan Sports Analytics Conference , 2020.[121] Pieter Robberechts. Valuing the art of pressing. In
StatsBomb Inno-vation in Football Conference , 2019.[122] Jiaxuan Wang, Ian Fox, Jonathan Skaza, Nick Linck, SatinderSingh, and Jenna Wiens. The advantage of doubling: a deep re-inforcement learning approach to studying the double team in thenba. arXiv preprint arXiv:1803.02940 , 2018.[123] Nathan Sandholtz and Luke Bornn. Replaying the NBA. In
Pro-ceedings of the MIT Sloan Sports Analytics Conference , 2018.[124] Hiroaki Kitano, Minoru Asada, Yasuo Kuniyoshi, Itsuki Noda, andEiichi Osawa. Robocup: The robot world cup initiative. In
Proceed-ings of the First International Conference on Autonomous Agents ,pages 340–347, 1997.[125] Karol Kurach, Anton Raichuk, Piotr Sta´nczyk, Michał Zaj ˛ac,Olivier Bachem, Lasse Espeholt, Carlos Riquelme, Damien Vin-cent, Marcin Michalski, Olivier Bousquet, et al. Google researchfootball: A novel reinforcement learning environment. In
Proceed-ings of the AAAI Conference on Artificial Intelligence , volume 34,pages 4501–4510, 2020.[126] Hongyao Tang, Jianye Hao, Tangjie Lv, Yingfeng Chen, ZongzhangZhang, Hangtian Jia, Chunxu Ren, Yan Zheng, ZhaopengMeng, Changjie Fan, et al. Hierarchical deep multiagent re-inforcement learning with temporal abstraction. arXiv preprintarXiv:1809.09332 , 2018.[127] David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Lau-rent Sifre, George Van Den Driessche, Julian Schrittwieser, IoannisAntonoglou, Veda Panneershelvam, Marc Lanctot, et al. Masteringthe game of go with deep neural networks and tree search.
Nature ,529(7587):484–489, 2016.[128] Ryan Beal, Georgios Chalkiadakis, Timothy J Norman, and Sar-vapali D Ramchurn. Optimising game tactics for football. arXivpreprint arXiv:2003.10294 , 2020.[129] Matthew Stephan and Kaan Koseler. A survey of baseball machinelearning: A technical report. Technical report, Miami University,2018.[130] Luca Pappalardo, Paolo Cintia, Paolo Ferragina, Emanuele Mas-succo, Dino Pedreschi, and Fosca Giannotti. Playerank: data-drivenperformance evaluation and player ranking in soccer via a machinelearning approach.
ACM Transactions on Intelligent Systems andTechnology (TIST) , 10(5):1–27, 2019.[131] Panna Felsen and Patrick Lucey. Body shots: Analyzing shootingstyles in the nba using body pose. In
Proceedings of the MIT SloanSports Analytics Conference , pages 3–4, 2017., pages 3–4, 2017.