Microsimulation Analysis for Network Traffic Assignment (MANTA) at Metropolitan-Scale for Agile Transportation Planning
11 Microsimulation Analysis for Network TrafficAssignment (MANTA) at Metropolitan-Scale forAgile Transportation Planning
Pavan Yedavalli
University of California, [email protected]
Krishna Kumar
University of Texas at [email protected]
Paul Waddell
University of California, [email protected]
Abstract —Abrupt changes in the environment, such as in-creasingly frequent and intense weather events due to climatechange or the extreme disruption caused by the coronaviruspandemic, have triggered massive and precipitous changes inhuman mobility. The ability to quickly predict traffic patternsin different scenarios has become more urgent to supportshort-term operations and long-term transportation planning,emergency management, and resource allocation. Urban trafficexhibits a high spatial correlation in which links adjacentto a congested link are likely to become congested due tospillback effects. The spillback behavior requires modeling theentire metropolitan area to recognize all of the upstream anddownstream effects from intentional or unintentional perturba-tions to the network. However, there is a well-known trade-offbetween increasing the level of detail of a model and decreasingcomputational performance. To achieve traffic microsimulationlevels of detail, current implementations often compromise bysimulating small spatial scales, such as intersections or corridorsthat ignore larger network dependencies. These simulators alsoeither require access to expensive high performance computingsystems or have computation times on the order of days orweeks that discourage productive research and real-time plan-ning. This paper addresses these performance shortcomings byintroducing a new platform, MANTA (Microsimulation Analysisfor Network Traffic Assignment), for traffic microsimulation atthe metropolitan-scale. MANTA employs a highly parallelizedGPU implementation that is fast enough to run simulations onlarge-scale demand and networks within a few minutes. Wetest our platform to simulate the entire Bay Area metropolitanregion over the course of the morning using half-second timesteps. The runtime for the nine-county Bay Area simulation isjust over four minutes, not including routing and initialization.This computational performance significantly improves the stateof the art in large-scale traffic microsimulation, and offersnew capacity for analyzing the detailed travel patterns andtravel choices of individuals for infrastructure planning andemergency management.
I. I
NTRODUCTION
Rapid global urbanization and an increase in the frequencyof extreme events, such as climate change-induced disruptive weather occurrences and global pandemics like COVID-19, are forcing us to re-examine the way we design andimprove the resilience of cities, including their transportationinfrastructure. Transportation simulation models offer theability to perform sensitivity analyses and ex-ante evaluationof the impact of potential infrastructure investments [1],[2], [3]. These simulations explore human mobility patterns,which are motivated by the need to engage in mandatoryand discretionary activities. They are carried out on variousmodes, including walking, biking, driving, or TNC services.The dynamics of traffic flow are affected by factors such asfrequency of trips, vehicle occupancy, length of the journeys,route choices, and driving speeds, producing congestion,traffic emissions, and an increase in traffic accidents [4]. Inaddition, certain transportation simulation, such as emergencyevacuation planning of a city in an extreme weather event,requires near real-time transportation planning. Hence, toaddress the need for regional-scale emergency scenarios andbroader infrastructure planning by policymakers and urbanplanners, we develop a fast metropolitan-scale traffic simu-lation engine capable of characterizing individual behaviors.Traffic modelers use three alternative types of traffic as-signment models to predict the impact of travel demandon the network: (1) macroscopic, (2) mesoscopic, and (3)microscopic, in decreasing order of traveler aggregation andincreasing order of granularity [5], [6]. Macroscopic modelsare based on the continuum assumption in classical fluidmechanics. The traffic flow is treated as continuous, similarto a flow of a liquid in a pipe, rather than that comprising ofdiscrete vehicles [7]. These macroscopic models are useful inanalyzing traffic systems covering a wide area, often acrossregions, and on highways where the overall speed dictatesthe macroscopic changes [1]. Unlike macroscopic modelsthat assume a continuous vehicular flow on the road link(edge), mesoscopic models employ aggregated volume delayfunctions, by clustering a set of vehicles into packets and a r X i v : . [ phy s i c s . s o c - ph ] J u l evaluating the movement of these clusters [1]. In contrast tothese models, microscopic traffic simulation models provideeven greater granularity, giving explicit consideration tothe interactions between individual vehicles within a trafficstream and employing characteristics such as vehicle lengths,speeds, accelerations, time, and space headways [8].Regional-scale transportation modeling has been domi-nated by the macroscopic and mesoscopic models, due totheir relative computational efficiency and familiarity [9].However, one of the significant drawbacks of these simulatorsis their lack of granularity. They are limited by the accuracyof representing real-world vehicle dynamics, especially incongested regimes and for emergency scenarios [8], [10].Traffic flow dynamics are naturally an outcome of the inter-action of a many-particle system, where each particle exhibitsdifferent characteristics [11]. Only a microsimulation modelcan capture these intricacies of individual components andcomplex interactions with reasonable accuracy [12], [11],[8], [13]. However, microsimulation has a high computa-tional cost due to the granularity required in simulating ofvehicle movements. Hence, regional-scale microsimulationhas generally been impractical [14], [1]. Although manytraffic simulators exist, such as MATSim, SUMO, AIMSUN,Polaris, TRANSSIM, VISSIM, and DynaMIT, among others,these simulators are not designed to tackle large-scale trafficmicrosimulation efficiently [15], [16], [17], [18], [19], [20],[21]. As a result, techniques such as sampling a small fractionof the transportation demand are currently employed toachieve regional-scale traffic models in a reasonable amountof time and computational cost.This paper introduces a massively parallelized GPU imple-mentation of a metropolitan-scale microsimulation engine -MANTA. MANTA is an agile regional-scale microsimulatorcapable of efficiently simulating over 7 million agents ata spatial scale as large as the San Francisco Bay Area, inunder 10 minutes. First, we present the components of thesimulation, then the mathematical theory and implementationof the simulator, followed by the results of a case study inthe Bay Area. We then present the calibration and validationof the simulator, performance benchmarks, limitations andfuture work, and finally the conclusions.II. C OMPONENTS
The objective of this study is to perform a regional-scale microsimulation of vehicular traffic of the Bay Area,incorporating individual trips on a typical workday morning.The microsimulator builds on the initial implementationby [2], [22]. In this section, the network generation, demandcreation, routing, and simulation architectures are describedin detail.
A. Street Network
For the case study, we use the San Francisco Bay Area,which includes nine counties. The street network is con-structed from the OpenStreetMap (OSM) network within thepolygonal hull of the counties in the metropolitan area using Fig. 1: The Bay Area Network with 224,223 nodes and549,008 edgesthe OSMnx library [23]. The network contains all roads in theBay Area, from large primary roads to tertiary streets. TheOSM network currently considers points representing curvesor bends in edges to be nodes, which is not topologicallyaccurate for network analysis [22], [23]. Hence, the networktopology is simplified by retaining only those nodes atintersections and dead-ends. The simplified network topologyresults in a fully-connected network, in which all nodes inthe network are connected to at least one other node in thenetwork. There are no hanging nodes without a path. Figure 1shows the full network with 224,223 nodes and 549,008edges. The number of lanes, length, and free-flow speedsfor each edge are then extracted from OSM data or imputed.The speed limit of each edge is taken from OSM if available;if not, a free-flow speed based on the edge’s number of lanes,and the type of road is derived and then used as the edge’sspeed limit. If the number of lanes is not available, then arecommended default value from OSM is used dependingon the type of road. For instance, a tertiary road without aspecified number of lanes is given a default speed limit of 20mph, and a motorway without a specified number of lanes isgiven a default speed limit of 57.5 mph.
B. Demand
The origin-destination (OD) demand is derived from datagenerated by the Bay Area Metropolitan Transportation Com-mission (MTC) travel model. The trips are narrowed tomorning trips between 5 AM and 12 PM and further restrictedto automobile trips, which consist of both driving and TNCtrips.The OD pairs are available at the granularity of trafficanalysis zones (TAZ). Thus, individual trips must be mapped to specific network nodes within the TAZ polygon. First,all the nodes of the street network are assigned to theirrespective TAZ polygon. After mapping nodes to their TAZs,each origin and destination is randomly assigned to one ofthe TAZ nodes. This process differs from [22], where eachorigin or destination is assigned to the centroid of the re-spective TAZ. Given that microsimulation models individualvehicle behavior, adding diversity of nodes to the simulation,rather than using the same node for every trip that containsthe same origin or destination TAZ, is expected to mimictravel patterns more realistically. The final OD demand has3,269,864 travelers.
C. Routing
After generating the network and the corresponding ODdemand, the next step of the simulation is to compute theshortest-path between each origin and destination. Shortest-path algorithms have been bottlenecks in many traffic mod-els, requiring either significant pre-processing time or greatcomputational cost [24]. The initial implementation of themicrosimulator used Johnson’s shortest-path algorithm, anall-pairs shortest-path (APSP) technique. APSP involvescomputing the shortest-path using Dijkstra’s algorithm for allpossible pairs of node connections in a network. This pro-duces a N × N matrix that must be stored in memory (RAM),where N is the total number of nodes in the network [2].However, the memory requirement grows significantly as N .For the Bay Area network with 224,223 nodes, the memoryrequired to compute APSP is 402 GB, making it extremelymemory intensive for accessible computing hardware. How-ever, Johnson’s algorithm remains valuable after this initialoverhead of calculating the all-pairs shortest-path, as futurequeries of the shortest-path would be a lookup with constanttime, O (1) .As a result, one of the significant contributions of thispaper is the integration of a parallelized Dijkstra’s pri-ority queue single-source shortest-path (SSSP) algorithm,described in [4], in which only the OD pairs required in thesimulation are computed. Typically, the number of agents n and their paths are significantly smaller than the APSP O ( N ) . The priority-queue algorithm is parallelized with ahybrid MPI/OpenMP scheme, which allows for linear scalingwith millions of agents. An open addressing scheme-basedhashmap is used to store key-value pairs of edge weights,hence updating the edge weights during the simulationand computing the shortest-path becomes more efficient.This open addressing scheme improves the performance ofhashmaps by 20%, providing quicker access to edges andconnectivity. A simulation with 3.2 million OD pairs routeson the large Bay Area network is calculated within 62minutes on an Intel I9 processor with 2 threads per core and14 cores per socket. This is a significant improvement fromthe prior APSP implementation, which could not be run dueto its massive memory requirements. Fig. 2: Departure times are chosen between 5 AM to 12PM to model the morning hours. It follows a Gaussiandistribution in which the bulk of the trips begin between 6:30AM and 8:30 AM. D. Microsimulation
The microsimulation framework we adopt is an enhancedand extended version of the architecture developed by [2].The vehicles move in discrete timesteps of δt = . seconds,following the state of the art microsimulators today [25]. Thesimulation described in this paper is carried out from 5 AM to12 PM to model a typical morning workday. Each traveler inthe OD demand is randomly assigned a departure time withinthis specified range by sampling from a normal distributionthat roughly mimics the morning peak-hour behavior, with apeak around 7:30 AM and a standard deviation of 45 minutes.The departure time specified for individual vehicles in thesimulation is presented in Figure 2.At each timestep, the vehicle’s travel time, position, andvelocity are updated. MANTA employs a unique traffic atlasconcept, akin to a texture atlas in the computer graphicscommunity or a discretization step in signal processing.Each road segment is assigned a contiguous set of bytes inmemory, where each byte represents t m meters of a laneand can be occupied by at most one vehicle [2]. This byteof data stores the car’s speed [2]. Hence, cars on the sameedge are located on adjacent bytes of memory. The trafficatlas significantly reduces the computational cost of findingneighboring vehicles, as it involves only looking up the statusof neighboring cells in memory, instead of a complex spatialdistance query, thus enabling GPU parallel computation. [2].In MANTA, the vehicular movement on an edge is dic-tated by conventional car following, lane changing, and gapacceptance algorithms [8]. The well-known Intelligent DriverModel (IDM), as shown in Equation (1), is used to controlthe vehicle dynamics through the network [26]. ˙ v = a (1 − ( vv o ) δ − ( s o + T v + v ∆ v √ ab s ) ) (1)where ˙ v is the current acceleration of the vehicle, a is themaximum possible acceleration of the vehicle, v is the currentspeed of the vehicle, v o is the speed limit of the edge, δ is Parameter Value Units a N (1 , ms b N (1 , ms T N ( . , ss N (1 , m TABLE I: IDM parameter ranges, derived from [26]the acceleration exponent, s is the gap between the vehicleand the leading vehicle, s is the minimum spacing allowedbetween vehicles when they are at a standstill, T is thedesired time headway, and b is the braking deceleration ofthe vehicle [2], [22]. The exact position of each vehicleat the current timestep is computed using this calculatedacceleration value ˙ v .The IDM contains several parameters that are calibratedfor the current case study using observed data. Simulationsfrom [26] were used to determine the ranges used in calibra-tion for a , b , T , and s , shown in Section II-D. In additionto car following, vehicles can also change lanes within anedge. There are two types of lane changes: mandatory anddiscretionary [2]. Mandatory lane changes occur when the ve-hicle must take an exit off the road, while discretionary lanechanges occur during overtaking or voluntary movements [2].The lane changing model gives the vehicle an exponentialprobability from switching from a discretionary lane changeto a mandatory lane change, as shown in Equation (2). m i = (cid:40) e − ( x i − x ) x i ≥ x x i ≤ x (2)where m i is the probability of a mandatory lane changefor vehicle i , x i is the distance of vehicle i to an exit orintersection, and x is the distance of a critical location tothe exit or intersection [27], [11]. Intuitively, as the vehicletravels further along in a path, its probability of making alane change to make a turn or exit increases. Once a vehiclehas decided to change lanes, the maneuver is performedif the lead and lag gaps are acceptable. The critical leador lag gap for a successful lane change are defined as theminimum distance to the following or lagging vehicle atwhich a lane change can be performed, respectively, as shownin Equation (3). g lead = max( g a , g a + α a v i + α a ( v i − v a )) + (cid:15) a (3) g lag = max( g b , g b + α b v i + α b ( v i − v b )) + (cid:15) b (4)where g lead is the critical lead gap for a lane change, g lag is the critical lag gap for a lane change, g a is the desiredlead gap for a lane change, g b is the desired lag gap for alane change, α is a system parameter (typically [0.05,0.40])that controls the gap based on speed, v i is the speed of thevehicle, v a is the speed of the lead vehicle, v b is the speed ofthe lag vehicle, and (cid:15) a and (cid:15) b are the random components [2].The representation and modeling of intersections in thisinitial application of the traffic simulator is simplistic and notrepresentative of diverse real-world dynamics at intersections.We consider two different types of traffic control. Case 1 Fig. 3: Histogram (log y-axis) showing the number of edgesthat see a particular vehicle count across the time rangesimulated. Most edges see fewer than 100 vehicles in thetimeframetraffic control is a flashing red light at each node, whereonly one vehicle can move into the intersection at a particulartime. If the node contains n inbound edges and m outboundedges, the system will create a round-robin of the nm combinations for all cars to pass through the intersectionbased on their position in their lane queue [22]. Case 2traffic control assigns every node as a green light, where allcars pass through with no delay. This is clearly not realisticfor most nodes that have stop signs or traffic lights, butis plausible for nodes along highway interchanges. Thesetwo different conditions produce surprisingly realistic overallresults despite the simplification in the intersection control.The results are discussed later in this paper. Future workincludes the ability to infer the correct traffic control at everyintersection using deep learning techniques.III. S HORTEST - PATH RESULTS
Preliminary travel patterns already emerge from the initialshortest-path calculation. Figure 3 displays a log histogramof edge volumes across the network, showing that the bulkof edges have vehicle counts below 1000 and are traversedinfrequently. Only a small fraction of the edges account forthe majority of the most frequently traversed routes. Routesacross the Bay Bridge are shown in Figure 4. Unsurprisingly,the Bay Bridge remains a unique outlier, as it producesthe maximum volume at 31270 vehicles in the seven-hourduration. From [28] by AC Transit and ARUP, 41727 tripsout of a total of approximately 4M trips traverse the BayBridge between 5 AM and 12 PM, representing 1% ofall trips. This proportion of Bay Bridge traversals matchesthe proportion from the microsimulation at roughly 0.98%(31270 trips out of 3.2M total trips in the Bay Area).IV. S
IMULATION RESULTS
Infrastructure and scenario planning requires a high degreeof accuracy in modeling the vehicle dynamics. This section
Fig. 4: All routes across the Bay Bridgehighlights the calibration and validation techniques alongwith the microsimulator results. Previous studies have reliedon vehicle counts, queue lengths at intersections, and vehiclespeeds at loop detectors as ground truth data for calibrationand validation [29]. In this work, we adopt novel approachesof calibration and validation using granular GPS trackingfrom Uber Movement data, which includes velocities ofdifferent edges over time.
A. Calibration
Traffic microsimulators require calibration to real-worlddata to adequately represent observed dynamics across a widerange of the network [17]. As previously mentioned, in theIDM, parameters a , b , T , and s are require calibration. Theobjective of the calibration process is to minimize the sum ofthe errors between every edge’s speed from MANTA and theUber data (L1 norm). This optimization problem is specifiedin Equation (5). min a,b,T,s N (cid:88) n =1 | (cid:80) Kk =1 ˙ v k tK − v uber | (5)where v = ˙ vt , a is the acceleration, b is braking, T is timeheadway, s is the linear jam distance, δ is the exponent of theIDM, K is the number of cars on each edge, N is the numberof edges that were successfully matched between Uber’s dataand MANTA data. Expanding further in Equation (6), min a,b,T,s N (cid:88) n =1 (cid:80) Kk =1 ˙[ a (1 − ( v k v ,n ) δ − ( s o + T v + v ∆ v √ ab s ) ] tK − v uber | (6)where, as previously specified, a is the acceleration potential, b is the braking potential, T is time headway, and s is thelinear jam distance.Given the highly nonlinear nature of the objective function,a numerical method is used. We constrain a and b to [1 , meters per second squared, T to [0 . , seconds, and s to Fig. 5: The calibration process: average mean differencebetween Uber and MANTA speeds over time [1 . , . meters, and set δ to 4, the standard exponent of theIDM [26]. A mini-batch gradient descent is then carried out,with each iteration executing runs for 5 different sets of a , b , T , and s . After simulating every new set, we gathered thesum of every edge’s delta in speed between MANTA andUber, represented by (cid:80) Kk =1 ˙ v k tK . The goal is to find the set a , b , T , s that minimizes this sum of differences. The setthat produces the lowest mean difference is chosen as thenominal vector for the next iteration. Each parameter is thenperturbed by a value chosen randomly in the range [ − , .The calibration process converges once the mean differencedecreases below a particular threshold. This threshold waschosen to be . miles per hour due to calibration runtimelimitations. As shown in Figure 5, the calibration processconverged after five iterations. B. Validation
Validation is performed for both the shortest-path algo-rithm and the traffic microsimulator. MANTA’s shortest-pathalgorithm is validated by comparing the routes against theCalifornia Household Travel Survey (CHTS) data for theBay Area [30]. Figure 6 presents the distances traveled byeach vehicle for both MANTA and CHTS. The distributionof distances is heavily right-skewed, suggesting that mosttrips are fewer than 25 km. While CHTS data are sparse(69000 trips versus 3.2M trips in MANTA), we can stillsee similarities. The mean distance traveled is 11.3 km inMANTA and 13.5 km in CHTS. Median values are 6.46 kmand 5.33 km in MANTA and CHTS, respectively. The 75thpercentile distances are also similar, at 13.6 km and 13.7 kmfor MANTA and CHTS, respectively.The validation of the traffic microsimulator involves com-paring the MANTA outputs to Uber Movement distributionsat specific time slices. In particular, using Q2 Uber Movementdata from 2019, 95,510 edges, or 17%, of the total edges arematched to the Uber network.One important enhancement is made to the simulator tobetter reflect vehicular behavior. In the IDM, v represents thefree-flow velocity of a vehicle on an edge, typically the speed Fig. 6: Comparison of trip lengths in MANTA versus Cal-ifornia Household Travel Survey data. Median distance inMANTA is 6.46 km and in CHTS is 5.33 km.limit of each edge from OSM or from a standard convention.However, in order to mimic the variance of driving patternsacross travelers, each driver’s maximum possible speed peredge, v , is sampled from a Gaussian distribution centeredaround the edge’s predetermined speed limit with a standarddeviation of σ s , where σ s is the standard deviation ofvehicle speeds at each speed limit s . Every vehicle thus hasa slightly different maximum allowable speed on each edgeit traverses.For the simulation run between 5 AM - 12 PM, the curvesdisplayed are for 5 AM - 6 AM, a less congested time period,and 8 AM - 9 AM, a more congested time period. Within eachtime period, a subset of curves is presented at different speedlimits. For instance, Figure 7a and Figure 7b show the speeddistributions from MANTA on edges with 35 mph comparedto the Uber Movement data on those same edges, between5 AM and 6 AM, and 8 AM and 9 AM, respectively. Thefigures confirm that both MANTA and Uber average speedsare higher between 5 AM and 6 AM than those between 8AM and 9 AM.Figure 8 shows the average speeds of MANTA and Uberacross all speed limits between 5 AM and 6 AM. At low-speed edges ( <
30 mph), MANTA simulation speeds areapproximately 5 mph slower than Uber’s real-world data.This indicates that the congestion effects are larger at lowerspeeds in MANTA. The Uber speeds also reflect that manydrivers tend to go above the speed limits more so on edgeswith lower speed limits than they do on edges with higherspeed limits. For edges with speed limits above 30 mph,MANTA and Uber estimates vary.The same plots are shown for the 8 AM - 9 AM timeframein Figure 9. Unlike in the 5 AM to 6 AM timeframe,MANTA simulation speeds are equal to or slower than Uber’sreal-world data across all speed limits. This indicates thatMANTA may be overly sensitive to congestion effects.Comparing the 5 AM - 6 AM timeslice with 8 AM - 9AM in MANTA, the average speeds estimated in the earlymorning time period in general are higher by 3 to 9 mph (a) 5 AM - 6 AM(b) 8 AM - 9 AM
Fig. 7: Kernel density plot comparing the MANTA and Uberdistributions at 35 mphacross all speed limits, with the greater differences being onedges with higher speed limits. This intuitively suggests thathigher speed roadways see less traffic at the early morninghours, and thus vehicles can travel at higher speeds due to thelack of congestion and lack of stoppage. However, roadwaysthat have lower speed limits do not allow for much higherspeeds regardless of the time of the day. This is likely due tothe presence of frequent intersections in the city. The Uberdata across the two timeslices also reflect this difference.
C. Red light / green light cases
In this study, a basic intersection model is adopted withtwo conditions, where every node is either a red light or agreen light. Between 5 AM - 6 AM, the average speed is17.5 mph, while the speed decreases to 12.9 mph in the 8AM - 9 AM timeslice, with each speed limit’s distributionshown in Figure 10. The reduction in speed between 8 AMand 9 AM suggests increased congestion, in comparison tothe free-flowing traffic in the early morning between 5 AM- 6 AM.When every node is a green light, the average speed acrossall speed limits from 5 AM - 6 AM is 24.5 mph, decreasing
Fig. 8: Average MANTA and Uber speeds across all speedlimits [5 AM - 6 AM]. The means and standard deviationsare shown in parentheses.to 17.8 mph from 8 AM - 9 AM, whose distribution acrossspeed limits is previously shown in Figure 9. The differencein speed limit between the early morning timeslice and 8AM - 9 AM time slice in the green light case is 4.6 mph,while in the red light condition, it is 6.7 mph. The deltasbetween the two timeslices, as well as the absolute speeds,highlight notable differences. Specifically, the average speedsin both timeslices under the red light condition is about 5mph lower than the green light condition. Such low speedsare unsurprising given that every vehicle must stop and waitits turn in the intersection queue.Notably, in Figure 9, the lower speed limits’ distributionstend to be right-skewed, following a lognormal pattern,while the distributions at higher speed limits become morecentered and follow a normal distribution. Snapshots of thesephenomena are shown in Figure 11 and Figure 12.The green light scenario was selected for calibration dueto its closer alignment with the Uber data as well as its moreaccurate representation of highway travel.V. P
ERFORMANCE BENCHMARKS
This section describes the profiling results of the twocomponents of MANTA: shortest-path and simulation. Fig. 9: Average MANTA and Uber speeds across all speedlimits [8 AM - 9 AM]. The means and standard deviationsare shown in parentheses.Fig. 10: Average MANTA speeds across all speed limits [8AM - 9 AM] in the red light case. The means and standarddeviations are shown in parentheses.
Fig. 11: Fit to lognormal distribution for 20 mph speed limitin green light scenario (case 2)Fig. 12: Fit to normal distribution for 45 mph speed limit ingreen light scenario (case 2)
A. Shortest-path benchmarks
In our network of approximately 225K nodes, 550K edges,and 3.2M OD pairs, the SSSP shortest-path algorithm carriesout the computation of all OD pairs’ routes in approximately62 minutes on a single node. Figure 13 shows the time-required to run up to 1 million agents on a distributed com-pute cluster utilizing both MPI and OpenMP parallelizationschemes. The routing algorithm shows strong scaling thatmatches the theoretical scaling up to 1024 cores for routing 1million agents. In comparison to existing routing algorithms,such as the heuristic-based Ligra [31] and iGraph [32],the priority-queue based Dijkstra is 2.2% and 55% faster,respectively, on a single node, as shown in Figure 14. Thepriority-queue Dijkstra algorithm also uses higher effectiveCPU usage of 94.1% with an average RAM usage of 4.81GB.
B. Microsimulator benchmarks
The computational performance of the MANTA simulatoris compared with Simulation for Urban Mobility (SUMO)and JDEQSIM, a single-thread alternative available in MAT-Sim, two well-known open-source simulators in transporta- Fig. 13: Time required to route agents using priority-queueDijkstra algorithm for the Bay-Area network on distributedcomputing environment (MPI + OpenMP) parallelization.Tests were run on 32 nodes with Intel Intel Xeon Skylake6142 processors.Fig. 14: Speedup of priority-queue routing algorithm forthe Bay-Area network on distributed computing environment(MPI + OpenMP) parallelization. Tests were run on 32 nodeswith Intel Intel Xeon Skylake 6142 processors.tion. The simulation of the Bay Area network and the demandbetween 5 AM - 12 PM are used for the comparison exercise.SUMO offers two options to build the network: one that con-tains internal links or lanes within intersections, and one thatdoes not contain internal links [16]. Considering MANTA’ssimplified intersection model, the SUMO model withoutinternal links is the most appropriate comparison. The SUMOmodel with internal links is also included for completenessbut is only relevant for future iterations of MANTA that willinclude more advanced intersection control.Section V-B shows the runtime comparison of MANTAagainst SUMO and JDEQSIM. The table also indicates whenthe results are linearly extrapolated, due to the inability tocomplete simulations in a reasonable time. Extrapolating thesimulation runtime linearly, MANTA performs nearly 27000x
Simulator Time (mins) Type
MANTA . FullSUMO meso simplified (MeS)
FullSUMO micro simplified (MiS)
Lin. extrap.SUMO meso advanced (MeA)
FullSUMO micro advanced (MiA)
Lin. extrap.JDEQSIM . Full
TABLE II: MANTA’s runtimes compared to SUMO andJDEQSIM. Full implies that the entire simulation was ableto complete. Lin. extrap. implies that only part of the sim-ulation was able to complete and the full time was linearlyextrapolated from this preliminary time.faster than SUMO. MANTA carried out the full microscopicsimulation of 3.2M trips at .5 s timesteps in 4.6 minutes,while SUMO’s simulator is estimated to take nearly 87 days,linearly extrapolated from the initial run of 194 minutes for5000 trips. SUMO also has a mesoscopic simulator, whichrequires approximately 29 hours (1740 minutes) for the BayArea simulation.A primary reason for such a dramatic difference in run-times is that typically SUMO uses a traffic assignment modelfor routing. When the routes are fixed, as in this example,SUMO sees undesired jamming, as many roads are not filledto their capacities while other roads are filled excessively. Theresulting congestion increases the simulation time in SUMO.While MANTA is not designed to produce an equilibriumassignment, it is able to manage fixed routes in a moremanageable way than SUMO due to the scalable trafficatlas. Notably, SUMO’s microsimulation does not supportparallelization; only the routing algorithm is parallelized,which is not germane for this comparison.JDEQSIM, developed at ETH Zurich, is an extension ofMATSim integrated with BEAM, the modeling frameworkfor Behavior, Energy, Autonomy, and Mobility, developed byLawrence Berkeley National Laboratory. JDEQSIM achievesa level of granularity between cellular automata and meso-scopic simulation using event-based dynamics. However, itdoes not model granular movements at the microsimulationlevel, such as lane changing. [6].Figure 15 shows the comparison of runtimes betweenJDEQSIM and MANTA. The JDEQSIM runtime is ap-proximately 6.6 minutes, on average over 50 runs, and iscomparable to MANTA’s runtime of 4.6 minutes. The GPUparallelized traffic microsimulation in MANTA is 43% fasterthan aggregated simulators such as JDEQSIM. In comparisonto the SUMO microsimulation, MANTA is several ordersof magnitude faster. Considering the finer level of behav-ioral granularity achieved by MANTA at the runtime of amesoscopic simulator, these results clearly demonstrate theapplicability of MANTA for regional-scale traffic microsim-ulations.Other parallel microsimulators exist as well, including[33], [34], [35], but they either require expensive super-computing machinery or carry out simulations on smallernetworks with longer computation times. Fig. 15: Simulator runtimes (log scale y-axis) across differentsimulators. MANTA performs slightly better than the par-allelized mesoscopic JDEQSIM and is on the same orderof magnitude. MANTA performs significantly better thanthe mesoscopic version of SUMO with either the simpli-fied (MeS) or advanced intersection modeling (MeA). Themicroscopic version of SUMO with simplified intersections(MiS) and advanced intersections (MiA) could not be runcompletely, and thus times were linearly extrapolated, reflect-ing that it would take tens of days to complete.VI. L
IMITATIONS AND ONGOING WORK
The traffic microsimulation in MANTA achieves signif-icant advances in computational performance using verylarge-scale networks and demand, but important limitationsremain. The first limitation is the use of simplified intersec-tion modeling. Although the simplification does not seem tohave excessively impacted the validation results compared toreal-world Uber Movement data, work is being conducted intraffic control inference using convolutional neural networksand vehicle trajectory data. We anticipate that accurate inter-section modeling will produce more precise travel times anda better representation of the vehicle dynamics.The second limitation is the demand profile. This workuses a synthetic Bay Area MTC 2017 travel model thatrepresents the daily demand in five large time blocks andcarries out a static traffic assignment. Future work involvestightly integrating MANTA with a dynamic travel demandmodel such as ActivitySim. The future versions of MANTAwill also include dynamic routing.VII. C
ONCLUSIONS
This paper presents a novel traffic microsimulator,MANTA, that addresses the challenges of accurate trafficmicrosimulation at the metropolitan-scale. MANTA is highlyefficient and is capable of simulating real-world traffic de-mand with a fine level of granularity on large-scale net-works. The runtime efficiency of MANTA is achieved byefficiently coupling a distributed CPU-parallelized shortest-path algorithm and a massively parallelized GPU simulationthat utilizes a novel traffic atlas to map the spatial distribution of vehicles as contiguous bytes of memory. The capabilityof MANTA is demonstrated by simulating a typical morn-ing workday of the nine-county Bay Area network with550K edges and 225K nodes, and approximately 3.2M ODpairs. The shortest-path calculations are completed in 62minutes, and a simulation of 7 hours from 5 AM to 12PM with .5 second timesteps is completed in 4.6 minutes.This is several orders of magnitude faster than the state ofthe art microsimulators with similar hardware. Achievingcompelling performance in both efficiency and accuracy,MANTA offers significant potential for fast scenario planningin both short- and long-term applications in metropolitanand regional-scale analysis. Ongoing enhancements includeimproved intersection control, incorporating dynamic trafficassignment, and tightly integrating activity demand.VIII. A CKNOWLEDGEMENTS
This report and the work described were sponsored bythe U.S. Department of Energy (DOE) Vehicle Technolo-gies Office (VTO) under the Systems and Modeling forAccelerated Research in Transportation (SMART) MobilityLaboratory Consortium, an initiative of the Energy EfficientMobility Systems (EEMS) Program. The following DOEOffice of Energy Efficiency and Renewable Energy (EERE)managers played important roles in establishing the projectconcept, advancing implementation, and providing ongoingguidance: David Anderson, Rachael Nealer, and Erin Boydas well as Prasad Gupte. This work was funded by the U.S.Department of Energy Vehicle Technologies Office underLawrence Berkeley National Laboratory Contract No. DE-AC02-05CH11231.The authors would like to give a special thanks to KenichiSoga, Bingyu Zhao, and the cb-cities research group at theUniversity of California, Berkeley and the University of Cam-bridge; Rashid Waraich, Artavazd Balayan, and the BEAMproject team at Lawrence Berkeley National Laboratory; andthe SUMO open-source team for remote simulation support.IX. A
PPENDIX
The source code for the microsimulator is available atMANTA, a module of the Urban Data Science Toolkit.R
EFERENCES[1] Z. Kokkinogenis, L. S. Passos, R. Rossetti, and J. Gabriel, “Towardsthe next-generation traffic simulation tools: A first evaluation,”
IberianConference on Information Systems and Technologies , p. 14, 2011.[2] I. Garcia-Dorado, D. G. Aliaga, and S. V. Ukkusuri, “Designing large-scale interactive traffic animations for urban modeling: Designinglarge-scale interactive traffic animations for urban modeling,”
Com-puter Graphics Forum , vol. 33, pp. 411–420, May 2014.[3] P. Waddell, “UrbanSim Modeling Urban Development for Land Use,Transportation, and Environmental Planning,”
Journal of the AmericanPlanning Association , 2002.[4] B. Zhao, K. Soga, and K. Kumar, “Agent-Based Model (ABM) forCity-Scale Traffic Simulation: A Case Study on San Francisco..”[5] R. T. Milam, M. Birnbaum, C. Ganson, S. Handy, and J. Walters,“Closing the Induced Vehicle Travel Gap Between Research and Prac-tice,”
Transportation Research Record: Journal of the TransportationResearch Board , vol. 2653, pp. 10–16, Jan. 2017. [6] R. A. Waraich, D. Charypar, M. Balmer, and K. W. Axhausen, “Perfor-mance Improvements for Large-Scale Traffic Simulation in MATSim,”in
Computational Approaches for Urban Environments (M. Helbich,J. Jokar Arsanjani, and M. Leitner, eds.), pp. 211–233, Cham: SpringerInternational Publishing, 2015.[7] S. Maerivoet and B. De Moor, “Transportation Planning and TrafficFlow Models,” arXiv:physics/0507127 , July 2005.[8] T. Toledo, H. Koutsopoulos, M. Ben-Akiva, and M. Jha, “MicroscopicTraffic Simulation: Models and Application,” in
Simulation Approachesin Transportation Analysis , pp. 99–130, New York: Springer-Verlag,2005.[9] G. Kotusevski and K. A. Hawick, “A Review of Traffic SimulationSoftware,”
Research Letters in the Information and MathematicalSciences , p. 20, 2009.[10] K. W. Axhausen and T. G¨arling, “Activity-based approaches to travelanalysis: Conceptual frameworks, models, and research problems,”
Transport Reviews , vol. 12, pp. 323–341, Oct. 1992.[11] Q. Yang and H. N. Koutsopoulos, “A Microscopic Traffic Simulatorfor evaluation of dynamic traffic management systems,”
TransportationResearch Part C: Emerging Technologies , vol. 4, pp. 113–129, June1996.[12] A. Loder, L. Amb¨uhl, M. Menendez, and K. W. Axhausen, “Under-standing traffic capacity of urban networks,”
Scientific Reports , vol. 9,p. 16283, Dec. 2019.[13] N. Geroliminis and A. Skabardonis, “Identification and Analysis ofQueue Spillovers in City Street Networks,”
IEEE Transactions onIntelligent Transportation Systems , vol. 12, pp. 1107–1115, Dec. 2011.[14] M. Saidallah, A. El Fergougui, and A. E. Elalaoui, “A ComparativeStudy of Urban Road Traffic Simulators,”
MATEC Web of Conferences ,vol. 81, p. 05002, 2016.[15] A. Horni, K. Nagel, and K. Axhausen, eds.,
Multi-Agent TransportSimulation MATSim . London: Ubiquity Press, Aug. 2016.[16] D. Krajzewicz, M. Bonert, and P. Wagner, “The open source trafficsimulation package SUMO,” in
RoboCup 2006 , RoboCup 2006, June2006.[17] J. Barcel´o and J. Casas, “Dynamic Network Simulation with AIM-SUN,” in
Simulation Approaches in Transportation Analysis (R. Kita-mura and M. Kuwahara, eds.), vol. 31, pp. 57–98, New York: Springer-Verlag, 2005.[18] J. Auld, M. Hope, H. Ley, V. Sokolov, B. Xu, and K. Zhang,“POLARIS: Agent-based modeling framework development and im-plementation for integrated travel demand and network and operationssimulations,”
Transportation Research Part C: Emerging Technologies ,vol. 64, pp. 101–116, Mar. 2016.[19] P. Saxena, D. Singh, M. Pant, and I. Giannoccaro, eds.,
ProblemSolving and Uncertainty Modeling through Optimization and SoftComputing Applications: . Advances in Computational Intelligence andRobotics, IGI Global, 2016.[20] B. B. Park and J. D. Schneeberger, “Microscopic Simulation ModelCalibration and Validation: Case Study of VISSIM Simulation Modelfor a Coordinated Actuated Signal System,”
Transportation ResearchRecord: Journal of the Transportation Research Board , vol. 1856,pp. 185–192, Jan. 2003.[21] M. Ben-Akiva, M. Bierlaire, H. Koutsopoulos, and R. Mishalani, “Dy-naMIT: A simulation-based system for traffic prediction,”
DACCORDShort Term Forecasting Workshop , p. 12, 1998.[22] P. Waddell, G. Boeing, M. Gardner, and E. Porter, “An IntegratedPipeline Architecture for Modeling Urban Land Use, Travel Demand,and Traffic Assignment,” arXiv:1802.09335 [cs] , Feb. 2018.[23] G. Boeing, “OSMnx: New methods for acquiring, constructing, analyz-ing, and visualizing complex street networks,”
Computers, Environmentand Urban Systems , vol. 65, pp. 126–139, Sept. 2017.[24] D. Delling, P. Sanders, D. Schultes, and D. Wagner, “EngineeringRoute Planning Algorithms,” in
Algorithmics of Large and ComplexNetworks: Design, Analysis, and Simulation (J. Lerner, D. Wagner, andK. A. Zweig, eds.), Lecture Notes in Computer Science, pp. 117–139,Berlin, Heidelberg: Springer Berlin Heidelberg, 2009.[25] R. Dowling, A. Skabardonis, and V. Alexiadis, “Traffic AnalysisToolbox, Volume III: Guidelines for Applying Traffic MicrosimulationModeling Software,”
Transportation Research Board , June 2004.[26] M. Treiber and A. Kesting,
Traffic Flow Dynamics: Data, Models andSimulation . Springer Berlin Heidelberg, 2013.[27] M. S. Iqbal, C. F. Choudhury, P. Wang, and M. C. Gonz´alez, “Devel-opment of origin–destination matrices using mobile phone call data,” Transportation Research Part C: Emerging Technologies , vol. 40,pp. 63–74, Mar. 2014.[28] AC Transit, Arup, and Cambridge Systematics, “Bay Bridge CorridorCongestion Study,” tech. rep., TJPA, 2010.[29] Technical Activities Division, Transportation Research Board, andNational Academies of Sciences, Engineering, and Medicine,
DynamicTraffic Assignment: A Primer
Proceedings of the 18th ACM SIG-PLAN Symposium on Principles and Practice of Parallel Programming ,pp. 135–146, 2013.[32] G. Csardi, T. Nepusz, et al. , “The igraph software package for complexnetwork research,”
InterJournal, complex systems , vol. 1695, no. 5,pp. 1–9, 2006.[33] C. Chan, B. Wang, J. Bachan, and J. Macfarlane, “Mobiliti: ScalableTransportation Simulation Using High-Performance Parallel Comput-ing,” in , (Maui, HI), pp. 634–641, IEEE, Nov. 2018.[34] J. Barcel´o, J. L. Ferrer, D. Garc´ıa, M. Florian, and E. L. Saux,“Parallelization of Microscopic Traffic Simulation for Att SystemsAnalysis,” in
Equilibrium and Advanced Transportation Modelling (P. Marcotte and S. Nguyen, eds.), pp. 1–26, Boston, MA: SpringerUS, 1998.[35] K. Nagel and M. Rickert, “Parallel implementation of the TRANSIMSmicro-simulation,”