Tracking Individual Targets in High Density Crowd Scenes Analysis of a Video Recording in Hajj 2009
RRunning title
TrackingIndividualTargetsinHighDensityCrowdScenes
July 2014
AdvancesinComplexSystems
Vol. XXI, No. 1
Tracking Individual Targets in HighDensity Crowd ScenesAnalysis of a Video Recording inHajj2009 M ohamed H. D ridi
Institute of Theoretical Physics 1
University of Stuttgart [email protected]
Abstract
In this paper we present a number of methods (manual, semi-automatic and automatic) for trackingindividual targets in high density crowd scenes where thousand of people are gathered. The necessary dataabout the motion of individuals and a lot of other physical information can be extracted from consecutiveimage sequences in different ways, including optical flow and block motion estimation. One of the famousmethods for tracking moving objects is the block matching method. This way to estimate subject motionrequires the specification of a comparison window which determines the scale of the estimate. In this workwe present a real-time method for pedestrian recognition and tracking in sequences of high resolutionimages obtained by a stationary (high definition) camera located in different places on the Haram mosquein Mecca. The objective is to estimate pedestrian velocities as a function of the local density.The resultingdata of tracking moving pedestrians based on video sequences are presented in the following section.Through the evaluated system the spatio-temporal coordinates of each pedestrian during the Tawaf ritualare established. The pilgrim velocities as function of the local densities in the Mataf area (Haram MosqueMecca) are illustrated and very precisely documented.Tracking in such places where pedestrian density reaches 7 to 8 Persons/m is extremely challengingdue to the small number of pixels on the target, appearance ambiguity resulting from the dense packing,and severe inter-object occlusions. The tracking method which is outlined in this paper overcomes thesechallenges by using a virtual camera which is matched in position, rotation and focal length to the originalcamera in such a way that the features of the 3D-model match the feature position of the filmed mosque. Inthis model an individual feature has to be identified by eye, where contrast is a criterion. We do knowthat the pilgrims walk on a plane, and after matching the camera we also have the height of the plane in3D-space from our 3D-model. A point object is placed at the position of a selected pedestrian. Duringthe animation we set multiple animation-keys (approximately every 25 to 50 frames which equals 1 to 2seconds) for the position, such that the position of the point and the pedestrian overlay nearly at everytime. By combining all these variables with the available appearance information, we are able to trackindividual targets in high density crowds. Keywords: Pedestrian dynamics, Crowd management, Crowd control, Objects tracking. 1 a r X i v : . [ c s . C V ] A ug unning title TrackingIndividualTargetsinHighDensityCrowdScenes
July 2014
AdvancesinComplexSystems
Vol. XXI, No. 1
1. I ntroduction C r owd simulation has found its way intocomputer science, computer visualiza-tions and the computer simulation oforiented building construction and crowd man-agement [1]. With continuously growing pop-ulation around the world and with enormousevolution in the different modes of transporta-tion in the last decade a lot of paper have ap-peared with increasing interest in modellingcrowd and evacuation dynamics. Thus thesimulation of pedestrian flows has become animportant research area. Pedestrian modelsare based on macroscopic or microscopic be-haviour.The evolution and design of any pedestriansimulation model requires a lot of informationand data.A number of variables and attributes arisesfrom empirical data collection and need to beconsidered to develop and calibrate a (micro-scopic) pedestrian simulation model.For this reason we used different toolsand developed different methods to collect themicroscopic data and to analyse microscopicpedestrian flow. It is very important to mentionthat the pedestrian data collection especially ina dangerous situation is still very much in its in-fancy. An aim of this study is to establish moreclearness and understanding about the micro-scopic pedestrian flow characteristics. Manual,semi manual and automatic image processingdata collection systems were developed. Manypublished studies show that the microscopicspeed obey a normal distribution with a meanof 1.38 m/second and a standard deviation of0.37 m/second. The acceleration distributionalso resemblances a normal distribution withan average of 0.68 m/ square second [2, 3, 4].For the evolution and development ofpedestrian microscopic simulation models, alot of data was collected with the help of videorecording and tracking of moving entities inthe pedestrian flow using the coordinates of thehead path was established through image pro-cessing. A large trajectory dataset has been re-stored. For the observation of pedestrian flows in public places a Sony camera was used. Thisobservation was in different places where thepilgrims perform their rituals. Many variablescan be gathered to describe the behaviour ofpedestrians from different points of view. Thispaper describes how to obtain variables fromvideo taking and simple image processing thatcan represent the movement of pedestrians (pil-grims) and its variables. Moreover in this workwe try to understand several parameters in-fluencing the pedestrian behaviour in riots orpanic situations.For obtaining empirical data different meth-ods were used, automatic and manual meth-ods. We have analysed video recordingsof the crowd movement in the Tawaf inMosque/Mecca during the Hajj on the 27thof November, 2009. We have evaluated uniquevideo recordings of a 105 ×
154 m large Matafarea taken from the roof of the Mosque, whereupto 3 million Muslims perform the Tawaf andSa’y rituals within 24 hours.Both Microscopic Video Data Collectionand Microscopic Pedestrian Simulation Modelgenerate a database called PedFlow database.The properties and characteristics that are capa-ble of explaining microscopic pedestrian floware illustrated. A comparison between averageinstantaneous speed distributions describingthe real world obtained from different methods,and how they can be used in the calibrationand validation of the simulation tools, are ex-plained.
2. R elated work
Typically, manual counting was performedby tally sheet or mechanical or electroniccount board to collect density and speed datafor pedestrian. Pedestrian behaviour studiesare collected by manual observation or videorecording in different public places like corri-dors side walks and cross walks. The effective-ness of the data (pedestrian speed) collectedon any observed area is strongly related tothe number of pedestrians in the flow. Therelationship between speed, flow, and pedes-trian density for a crowd population or human2unning title
TrackingIndividualTargetsinHighDensityCrowdScenes
July 2014
AdvancesinComplexSystems
Vol. XXI, No. 1group has been published in many fundamen-tal diagrams developed by Fruin [5] and others[6]. Though for many reasons the method hasbeen used to detect and count vehicles in auto-matic way cannot be used to detect pedestrians,since this system has been evaluated throughpneumatic tube or inductance loops. As wecan deduce from later work on this technol-ogy the possibility of applying this method toreproduce trajectory and motion prediction isstill in a discussion phase.Other approaches use a neural networkframework recursively to predict pedestrianmotion and trajectory [7]. However the pedes-trian trajectories in this system are calculatedwith incorrect simplifications. In particular,only the nearest neighbour trajectories are con-sidered. The main shortcoming of such anestimation is that there is no uncertainty in thisprediction, moreover a comparison of differentpath prediction shows this is still far from thereality in order to predict that all objects willfollow the same set of paths exactly.A method which allowed people count-ing based on video texture synthesis and toreproduce motion in a novel way was intro-duced by Heisele and Woehler [8]. The methodworks under the assumption that people canbe segmented from the moving backgroundby means of appearance or motion properties.The scene image is clustered based on the colorand position (R, G, B, X, Y) of pixel. The ap-pearance of each pixel in a video frame is mod-elled as a mixture of Gaussian distributions.A algorithm is used that matches a sphericalcrust template to the foreground regions of thedepth map. Matching is done by a time de-lay neural network for object recognition andmotion analysis.A significant task in video intelligence sys-tems is the extraction of information about amoving objects e.g. detecting a moving crowdwith PedCount (a pedestrian counter systemusing CCTV) was developed by Tsuchikawa [9].It extracts the object using the one line path inthe image by background subtraction to makea space-time (X-T) binary image. The direc-tion of each travelling pedestrian is realized by the attitude of pedestrian region in the X-T im-age. They reported the need of background im-age reconstruction due to image illuminationchange. An algorithm to distinguish movingobject from illumination change is explainedbased on the variance of the pixel value andframe difference.
3. A nalyse of the V ideo T aking in H ajj The electronic and digital revolution in videotechniques during recent years has made it pos-sible to gather detailed data concerning pedes-trian behaviour, both in experiments and inreal life situations [10, 11, 12]. The big chal-lenge is to develop a new efficient method ofdefining and measuring basic quantities likedensity, flow and speed. Basic quantities ofpedestrian dynamics are the density ρ [1/m ]in an area A and the velocity (cid:126) v [m/s] of personsor a group of persons, and the flow through adoor or across a specific line (cid:126) Q = (cid:126) v ( (cid:126) r , t ) ρ ( (cid:126) r , t ) [1/s]. The measurements also yield mean val-ues of these quantities. The task is to improvethe given methods such that they allow to gofairly close to the real data of the crowd quanti-ties. The methods presented here are based onvideo tracking of the head from above. Notethat tracking of e.g. a shoulder or the chestmight be even better, though more difficult toobtain.The density distribution knowledge in avery crowded area allows us to draw a so calleddensity map to show us congestion directly asregions of high density. The relationship be-tween the pedestrian density ρ and the pedes-trian maximum walking speed v max are formal-ized into a graph known as the fundamentaldiagram v max = f ( ρ ) [3]. Since pedestriansmove slower in a region of high density, thesimulated particles should update their speedwith the surrounding circumstances to maxi-mize their rate of progress towards their goals.3unning title TrackingIndividualTargetsinHighDensityCrowdScenes
July 2014
AdvancesinComplexSystems
Vol. XXI, No. 1
Tawaf observations at the Haram mosque inMecca were made during Hajj 2009 by Mr.Faruk Oksay. The Mataf area has 10 entrances/ exits. The flow of the Tawaf is controlled.All pilgrims begin and end their Tawaf at thesame place (see fig.1). The number of pilgrimsduring this period is sufficient to observe thebehaviour of high density crowd dynamics.
Figure 1:
Overview of the Mosque with entrance doors.
Figure 1. shows the main gate doors, sideentrances, stairs to the Mataf open air of theHaram.All observations took place on FridayNovember 27th 2009 corresponding to 10th ofDhu al-Hijjah 1430 Hijri in the afternoon. Dur-ing the total observation period of three hours,three prayers (Midday, Asr and Maghreb(sunset-prayer)) were performed, where in thistime the Mataf area comes to a standstill (seefig. 2). Our video observations show that thepilgrims have the desire to be near the Kaaba.Therefore approximately 70 percent (visuallydetected on video) of the pilgrims performtheir Tawaf movement near the Kaaba wall,which causes a high density in this area.
Figure 2:
Pilgrims performing the prayer ritual in theMosque in Mecca. During performance ofprayer the Tawaf come to standstill, there areno movement around the Kaaba.
In Figure 2, one can see all of the pilgrimsperform the prayer ritual in the holy mosquein Mecca.The Tawaf around the Kaaba is a periodicmovement for the time between two prayers.The observed number of pilgrims performingtheir Tawaf ritual at the Mataf area increasesslowly after every prayer until the Mataf attainsit’s maximum capacity (see fig. 3).
Figure 3:
Mean velocity of pilgrims in the Mataf area asa function of time over 12 hours.
Figure 3 shows a typical pedestrian move-ment in the Mataf area over daytime. Duringprayer times individuals stand still and there-fore movement equals approximately zero. Thefluctuations in the velocity flow are created by4unning title
TrackingIndividualTargetsinHighDensityCrowdScenes
July 2014
AdvancesinComplexSystems
Vol. XXI, No. 1the turbulence in the pedestrian flux. Notethat the average local density in a specific loca-tion in the Mataf area exceeded 8 persons/m during the Hajj periods (see fig. 6 and 7). Our first goal is to identify new methods andcreate a test system capable of extracting pedes-trian movement information from video, sim-ilar to that collected by our HD-Cameras inthe Hajj-2009, such that any movement can beanalysed to spot suspicious activity. This taskto collect pedestrian data and extract pedes-trian motion from video sequences required aninvolvement and development of appropriatemethods, followed by further analysis of thisdata to identify emergent motion or crossingtrajectories.The secondary goal is to identify the limita-tions of the approach including the system anddata requirements for the techniques to workmore effectively. More specific, the projectgoals are: • Develop a framework for video and im-age analysis, • Develop an approach and relevant diag-nostic software to collect movement datafrom video, • Identify the requirements for such meth-ods to work effectively, such as imagequality, resolution and orientation, • Identify how to interpret movement in-formation, • Interpret the movement data and exam-ine abnormal behaviour, • Design and produce a working imple-mentation that demonstrates the abovegoals, • Identify approaches that could furtherimprove the system.
4. E stimation of C rowd D ensity There are different techniques developed toextract information describing the position ofpedestrians in a location, but not all of them areappropriate for detecting and pursuing pedes-trian movement under different and extremelyweather conditions. In their published work[13], Papageurgiou and Poggio developed asystem attempting to recognize human fig-ures based on pixel similarities through a largetraining set of figures under various light andweather conditions. To identify the movementof the figures, the system analyses the simi-larity between matches of consecutive frames.This method works quite well when the train-ing set is large, but requires a high compu-tational efficiency which achieves processingrates of 10 Hz [13]. The study shows that accu-rate recognition can be done with coarse imagedata.Another approach to estimate crowd den-sity is based on texture analysis. Velastin etal. [14] assumed that crowds with high den-sity possess texture properties. The proposedmethod, texture features were computed forthe whole image and applied to crowd densityestimation [15]. In particular, all displayed tex-tures, like wavelets [16, 17] and the gray leveldependence matrix [18, 19], were used to esti-mate crowd density. The results exhibit, howeffective statistical analysis of texture displayis compared to neural networks when measur-ing crowd density. Unfortunately, this systemexamines only static images and cannot covercrowd motion, but the techniques can be usedto track pedestrian movements.Other strategies based on image segmen-tation were pursued by Heisele and Woehler[8], where raw data is filtered to split the im-age into segments, which are then analysed.Those images that match particular shapes areanalysed further. This approach allows to dis-tinguish different images with common colorand luminescence. 5unning title
TrackingIndividualTargetsinHighDensityCrowdScenes
July 2014
AdvancesinComplexSystems
Vol. XXI, No. 1
Figure 4:
The Haram piazza top view (observation area).
In order to collect pedestrian data and to study pedestrian traffic flow operations ona platform in detail, observations were alsomade from a platform of the Haram Mosque inMecca. These observations concerned pilgrimwalking speeds and density distributions onthe Mataf area and (individual) walking timesas functions of the distance from the Kaabawall.
The estimation of crowd density is an impor-tant criterion for the validation of our simula-tion tools. Processing is done in three levels. • Existing footage is loaded on a 3D pro-gram as a backplate. • From several provided 2D- architecturaldrawings we build a 3D model of themosque. • A virtual camera has to be matched inposition, rotation and focal length to theoriginal camera so that the features ofthe 3D-model match the features of thefilmed mosque. As the dimensions of themosque are known, we then establish agrid of regular cells on the Mataf area,each one of which has a size of 5mx5m(see fig. 5). Through image editing soft-ware, we start a manual counting process.This regular grid is used to observe thedensity behaviour over all of the Matafarea, from the nearest range to the Kaabawall up to outside of the Mataf and theaccumulation process (by the Black Stoneand Maquam Ibrahim). The results ofthis investigation are shown in figures 6(a), (b), (c) and (d) and illustrate us thebehaviour of the pilgrim density on theMataf area at different times during theday.6unning title
TrackingIndividualTargetsinHighDensityCrowdScenes
July 2014
AdvancesinComplexSystems
Vol. XXI, No. 1
Figure 5:
Grid of regular cells with dimension of5m × With a new computer algorithm developedwithin this investigation, where the Mataf areais divided in regular cells. The number ofpedestrians in every cell as function of timeis determined through repeating the countingprocess many times. The average value is iden-tified as local density ρ ( (cid:126) r , t ) . The data extractedfrom the videos allowed us to determine notonly densities in larger areas, but also localdensities, speeds and flows. As an examplethe density distribution on the Mataf area isshown in figure: 6. The data was obtained bysemi-manual evaluation. Dependence of the Density distribution onthe Mataf as function of time
Figure 6shows density decline curves for different dis-tances from the Kabaa in a specific time. Thecurves indicate that the local density amountvary strongly over the (0 < x < 40 m) range.Figure:7 shows the pedestrian density dis-tribution on the Mataf area as a function of theposition (cid:126) r and time t . One clearly recognizesdensity waves, with maximum density pack-age near the Kaaba wall. There the averagelocal density can reach a critical value of 7 to8 persons/m . The congested area increasesthe local density to a critical and dangerousamount. As a consequence the pedestrians be-gin to push to increase their personal spaceand create shock-waves propagating throughthe crowd, which can be seen as density waves,or density packages. The Density map illustrates how the pedes-trian density decreases from the inside to out-side of the Mataf area, (see fig.8). As we havementioned that in the Mataf area pedestriansmove in the restricted space, the layout is grad-ually painted in different colors. The color ofevery point of the space corresponds to thecurrent density in this particular area. The den-sity map is constantly repainted according tothe actual values: when the density changes insome point, the color changes dynamically toreflect this change. In case of zero density thearea is not painted at all (see fig. 8 (a), (b), (c)and (d)).During the rush hour in a Hajj period thelocal density in the Mataf area reaches the max-imum as we can see in the following figures7 (d) and 8 (d). The local density can reach 8to 9 persons/m in a specific time during theday. The maximal density concentrates nearthe Kaaba wall. Densities over time and space
We observethe density behaviour on the Mataf area at dif-ferent times during the day, before and afterthe prayer, and we compare this density withthe simulation density results. The maximumregistered density was 7 to 8 persons/m andthis represents a high crowd density. The re-sults of the estimation based on the statisticalmethod, presented in figures 6,7 and 8, reacheda mean of 92 percent correct estimations. It ispossible to verify that the results were quitegood for all evaluated images except for theone made up of high density crowd images,which reached only 84 percent correct estima-tions. In the Mataf area, near the black stone,the pilgrim density reached over 9 persons/m .For this reason it is very difficult to recognizeand track every head and as a result, a 100 per-cent correct estimation would be very difficult.All statistical results illustrating the densitydistribution at the Mataf area at different timeintervals are demonstrated in the figure 9. 7unning title TrackingIndividualTargetsinHighDensityCrowdScenes
July 2014
AdvancesinComplexSystems
Vol. XXI, No. 1 D en s i t y [ p / m ] Linear Distance From The Kaaba Wall [m](a) 0 1 2 3 4 5 6 7 8 0 5 10 15 20 25 30 35 40 D en s i t y [ p / m ] Linear Distance From The Kaaba Wall [m](b) 0 1 2 3 4 5 6 7 8 0 5 10 15 20 25 30 35 40 D en s i t y [ p / m ] Linear Distance From The Kaaba Wall [m](c) 0 1 2 3 4 5 6 7 8 0 5 10 15 20 25 30 35 40 D en s i t y [ p / m ] Linear Distance From The Kaaba Wall [m](d)
Figure 6:
Decrease of pedestrian density on the Mataf Area as function of the distance from the Kaaba wall: (a) beforeMid-Day prayer; (b) shortly after Mid-Day Prayer; (c) half-hour after Mid-Day Prayer; (d) Rush Hour.
Figure 9:
Crowd density on the Mataf area in differenttime intervals. Highest density in the area ofthe Kaaba.
This part of the dissertation considers the roleof automatic estimations of crowd density andtheir importance for the automatic monitor-ing of areas where crowds are expected to bepresent. A new technique is proposed whichis able to estimate densities ranging from very low to very high concentrations of people. Thistechnique is based on the differences of texturemuster on the images of crowds. Images of lowdensity crowds exhibits rough textures, whileimages with high densities tend to present finertextures. The image pixels are classified in dif-ferent texture classes, and statistics of suchclasses are used to estimate the number of peo-ple. The texture classification and the crowddensity estimation are based on self-organizingneural networks. Results obtained estimatingthe number of people in a specific area of theHaram Mosque in Mecca are presented in fig-ure 10).8unning title
TrackingIndividualTargetsinHighDensityCrowdScenes
July 2014
AdvancesinComplexSystems
Vol. XXI, No. 1
Figure 7:
The Density Distribution on the Mataf Area ρ ( (cid:126) r , t ) with (cid:126) r = ( x , y ) . The figure also shows the densityindex (persons/m ). (a) before Mid-Day prayer (t = tbefore Mid-Day); (b) shortly after Mid-Day Prayer (t =tshortly after Mid-Day); (c) half-hour after Mid-Day Prayer (t = thalf-hour after Mid-Day); (d) Rush Hour(t = tRush). Figure 10:
Density distribution in the Mataf area beforeMid-Day prayer. Red color indicates highdensity, where green color indicates low pil-grims density.
In the latter paragraphs we focus on crowddensity estimation for several reasons. Accord-ing to the crowd disasters study by Helbingand Johansson [20], one of the most impor-tant aspects to keep a crowd safe is to predictand identify areas with high density crowdspreventing large crowd pressures to be builtup. Areas where crowds are likely to buildup should be identified prior to the event oroperation of the venue. This is important ascrowds usually exist in certain areas or at par-ticular times of the day. Places where crowddensity rises up over time are likely to congestand need careful observations to ensure thecrowd safety. Basically, crowd density surveil-lance and estimation can be a good solution formanagement and controlling the crowds safety.The results of the estimations obtained dur-ing the tests allow us to consider both meth-9unning title
TrackingIndividualTargetsinHighDensityCrowdScenes
July 2014
AdvancesinComplexSystems
Vol. XXI, No. 1 (a) 6 5 4 3 2 0 5 10 15 20 25 30 35 40x [m] 0 5 10 15 20 25 30 35 40 y [ m ] y [ m ] y [ m ] y [ m ] Figure 8:
The Density map indicates that the highest pedestrian density in the area of the Kaaba: (a) before Mid-Dayprayer; (b) shortly after Mid-Day Prayer; (c) half-hour after Mid-Day Prayer; (d) Rush Hour. ods successfully. While the statistical methodreached quite good estimation rates (around 92percent) for most groups, the spectral methodillustrated small deviations between the bestand the worst estimations, reaching on aver-age almost the same rates of correct estimationobtained by the statistical method.
5. M ethod of getting thepedestrian speed
As speeds are hard to observe, walking timeswere measured, from which walking speedswere derived. In addition to walking times andpedestrian densities other variables needed tobe considered to complete the input of the sim-ulation model (such as the number of in andout going pilgrims and the configuration ofthe structure during the rush hour at the Hajjperiod). The observables are the walking time,velocities and the corresponding densities ofthe pilgrims performing their Tawaf and Sa’y. The movements of the pilgrims going in andout of the Haram give us data to calculate theflux related to the Tawaf. The distribution ofboth in and out going pilgrims over the Haramcan be derived from this data. The secondtype of observation concerns individual walk-ing times. In order to measure the pilgrims’walking times in and out of the Haram, pil-grims were recorded from the moment theystarted walking from one spot to another, ei-ther on the piazza or going up the stairs. Thestart and duration of activities, such as Tawafor Sa’y, were measured also. Finally, locationsof origin, destination and possible activities ofthe pilgrims were registered. To do this, thepiazza is divided into small areas with a lengthof 5 × TrackingIndividualTargetsinHighDensityCrowdScenes
July 2014
AdvancesinComplexSystems
Vol. XXI, No. 1Predtetschenski-Milinski [1].
Data was collected on a specific subject groupof pedestrians who appeared to be 40 yearsof age or older. On the roof of the Mataf areawe selected our tracking subjects, consisting ofadult men, women and people in wheelchairs.The following individuals were specifically notconsidered: • Children under 13 years of age, • Pedestrians carrying children, heavybags, or suitcases, • Pedestrians holding hands or assistingothers across the Mataf, • Pedestrians using a quad pod cane,walker, two canes, or crutches.To accurately quantify the normal walkingspeeds of the various subject groups, pedes-trians who exhibited any of the following be-haviour were also not considered: • Crossing of the Mataf path diagonally, • Stopping or resting in the Mataf area, • Entering the roadway running (anythingfaster than a fast walk),The pedestrian sex (male or female) of eachindividual in the Mataf area was recorded, aswell as whether he or she was walking alone orin a group. The group size was also notedwhen applicable. A group was defined bytwo or more pilgrims walking the Mataf tra-jectory at about the same time, regardless ofwhether or not they were apparently friendsor associates. In the Mataf area, the pedestriangroups can reach 30 pilgrims walking togetherin the pedestrian stream. In addition, subjectspaths were monitored to determine when theystarted and ended their Tawaf. Being inside theMataf was defined as being within or on thepainted Tawaf walking lines. Other pedestrianbehaviour was recorded when if occurred: • Confusion (hesitation, sudden change indirection of travel or change of point ofinterest) exhibited before walking, • Confusion exhibited after entering theMataf trajectory, • Cane use, • Following the lead of other pedestrians, • Stopping in the walking path during theTawaf movement, • Difficulty going into Mataf, • Difficulty going out of the Mataf.Several methods were developed to check theaccuracy and performance of walking speedestimation abilities of the observers. First, thewalking speed was measured at the same timeby three observers, then correlations betweenthe estimates of all observers were determined.In particular, the walking velocity of one pil-grim was measured by the three observers andthe mean value was taken. The results of theseverification procedures are discussed after thenext section.
From our video recordings we choose placesbetween two minarets as references, (see fig.11). As the dimensions of the mosque wereknown, we then established a grid of regularcells covering all of the Mataf area, each onehaving a size of 5mx5m (see fig. 12). Thedistance between the two minarets is known.Pedestrian crossing times were measured witha digital timer and an electronic stopwatch wasimplemented and synchronized with the timerof the video recorder. The watch was startedas the subject stepped off the first minaret andstopped when the subject stepped out on theopposite minaret after crossing all the distancebetween the two minarets. 11unning title
TrackingIndividualTargetsinHighDensityCrowdScenes
July 2014
AdvancesinComplexSystems
Vol. XXI, No. 1
Figure 11:
Overview of the piazza of the Haram, theplace where our observations are made. Witha digital clock the individual walking times t p are measured. Since the distance between thetwo minarets is known from the architecturalplan of the Haram, the average of the localpedestrian velocities v ( (cid:126) r , t ) = (cid:107) (cid:126) v ( (cid:126) r , t ) (cid:107) canbe determined. From the roof of the Mosque every pedestriancan be identified. To establish the ability ofthe field observers to identify the fitness levelor the age of pedestrians with high accuracy asimple verification procedure was performed.The age estimation and the level of fitness ofthe pedestrians was based on their walkingspeed. It is a physio-medical fact that olderpedestrians walk more slowly than youngerones (this is easily supported by field data),however, the published or already existing dataon walking speeds and start-up times (i.e. thetime from the beginning of a Tawaf movementuntil the pedestrian steps off the Mataf) havemany shortcomings. Here we consider thecomplicated movement of the Tawaf and thehuman error rate of the observer. The walk-ing speed on the Mataf area can be affectedby many factors, one of the relevant factors isthe age of the pedestrian. This demonstratesthat the observations were quite good at iden-tifying older pedestrians or pedestrians withfitness deficiency or physical health problems. A digital stopwatch was integrated with thevideo recording sophisticated for the measure-ments of pedestrian crossing times. The cross-ing times of the same pilgrims were measuredduring five rounds of the Tawaf and the aver-age value was determined.
Figure 12:
Grid of regular cells with dimension of5m × p are determined and the average ofthe local speeds v ( (cid:126) r , t ) = (cid:107) (cid:126) v ( (cid:126) r , t ) (cid:107) is cal-culated. The average walking speed for malepedestrians is 1.37 m/s, female 1.22 m/s andfor people moving on wheelchairs 1.534 m/s. This research also examined the impact of thebuilding layout on the pedestrian speed distri-bution and the pedestrian density of pilgrimsperforming the Tawaf movement around theKaaba. The set of data of pedestrian walkingspeeds which were obtained through analysingvideo recording using a set of statistical tech-niques are displayed in figures 13 (a), (b) and(c). The results revealed that walking speedseems to be following a normal distribution nomatter of male, female, older or younger. Theaverage speed of young people is dramaticallylarger than that of older people, and the aver-12unning title
TrackingIndividualTargetsinHighDensityCrowdScenes
July 2014
AdvancesinComplexSystems
Vol. XXI, No. 1
Figure 13:
Walking speed distribution in Tawaf movement. (a) Women, (b) men, (c) wheelchairs and (d) shows acomparison of the three distributions. The average walking speed for female pedestrians is µ = m/swith a standard deviation of σ = while for male pedestrians µ = m/s and σ = and forpilgrims on wheelchairs µ = m/s and σ = . age speed of male is slightly larger than thatof female. The width of the obtained curves isrelated to the different standard deviations.The mean computed walking speed repre-sents the speed that 85 percent of pedestriansdid exceed. A total of 250 pedestrians wereobserved. Included were 100 male pedestriansof about 60 years of age, 100 women pedestri-ans and 50 wheelchair pedestrians. This datadescribes all of the pedestrians observed: thosewalking in the center of the stream and thosewalking by the edge of the Mataf trajectory.As is subsequently described, those who werewalking by the edge of the Mataf tended towalk more quickly. All observed pedestriansmoved in a rotational motion around the Kaaba counter-clockwise (Tawaf), in compliance withthe pilgrim stream.The mean walking speed for male pedes-trians was 1.37 m/s and 1.22 m/s for femalepedestrians. In conjunction with pilgrims old,the mean walking speed for younger pedestri-ans was 1.48 m/s and 1.20 m/s for older malepedestrians. The results revealed that the aver-age walking speed for young women are 1.32m/s and 1.12 m/s for old women. This means • Young male pedestrians had the fastestmean walking speeds [1.48 m/s] andolder females had the slowest [1.12 m/s].The differences between young men andyoung women [0.16 m/s] and between13unning title
TrackingIndividualTargetsinHighDensityCrowdScenes
July 2014
AdvancesinComplexSystems
Vol. XXI, No. 1older men and older women [0.1 m/s],this result shows a little deviation thatcan be traced back to the fitness levelof pedestrian or other factors, in thenormal condition are approximately thesame. The mean walking speed for theyounger pedestrians ranged from 1.37 to1.57 m/s across all conditions, with anoverall mean speed of 1.48 m/s. Themeans for the older pedestrians rangefrom 0.97 m/s to 1.26 m/s, with an over-all mean speed of 1.18 m/s. For designpurposes a mean speed of 1.33 m/s ap-peared appropriate; • Locations by the edge of the Mataf hadfaster walking speeds because such lo-cations has a lower pedestrian density.It is clear that the pedestrians near theKaaba had a short walk path but in thisplaces densities of 7 to 8 persons/ m canbe exceeded, making the movement ofpilgrims very slow and turbulent; • Places situated further away from theKaaba wall also tended to be associatedwith faster walking speeds. It is knownfrom other fundamental diagrams, thatpedestrians tend to walk faster along afree walkway. As might be expected thewalking speeds associated with variousfactors. The motion of a single individ-ual at any given time and the directionand speed result in a long list of possible(and very likely conflicting) forces andcircumstances.The data taken show that each of the loca-tions and surrounding factors have a significanteffect on the behaviour and walking speed ofthe pilgrims on the Mataf area, not forgettingthat the age of the pedestrians play a significantrole on the Tawaf movement and density peaksand jams are caused by pilgrims of age 70 andmore. For approximately one half of the loca-tion, the factors examined there also showedan important correlation between pedestrianage, the location and the mean walking speedof the pilgrims. This funding is consistent withresults published by Knoblauch [2]. The walking speed of pilgrims shows sta-tistically significant variations across a varietyof sites, times and environmental conditions(pedestrian density on the Mataf area). On theroof of the Mosque the pilgrim density is lowand every pedestrian can walk with his desiredvelocity. However, the mean walking speeddata is explicit by clustered for both pedestri-ans sex, men and women, independent of theage of the pilgrims are considered.
There exist numerous methods that track themovement of single individuals by inspectingtheir orientation and limb positions.This section highlights a real-time systemfor pedestrian tracking from sequences of highresolution images acquired by a stationary(high definition) camera. The objective wasto estimate pedestrian velocities as a functionof the local density. With this system the spatio-temporal coordinates of each pedestrian duringthe Tawaf ritual were established. Processingwas done through the following steps: • Existing footage was loaded onto a 3Dprogram as a backplate. • From several provided 2D- architecturaldrawings, a 3D model of the mosque wasbuilt. • A virtual camera was matched in posi-tion, rotation and focal length to the orig-inal camera so that the features of the 3D-model matched the features positionedon the filmed mosque. • Individual features were identified byeye, contrast is the criterion • We do know that the pilgrims walk on aplane, and after matching the camera wealso obtained the height of the plane in3D-space from our 3D model. • A point object was placed at the positionof a selected pedestrian. During the an-imation we set multiple animation-keys14unning title
TrackingIndividualTargetsinHighDensityCrowdScenes
July 2014
AdvancesinComplexSystems
Vol. XXI, No. 1(approx every 25 to 50 frames (equals 1to 2 seconds)) for the position, so that theposition of the point and the pedestrianoverlay nearly all the time. • By evolving the point with time we ob-tained the distance travelled, by measur-ing the distance from frame to frame.We also knew the time elapsed from thespeed per frame, and hence the speedcould be calculated.
Figure 14:
Pilgrims’ walking speeds on the edge of theMataf. The average walking speed is 1.0816m/s.
Figure 15:
Pilgrims’ walking speeds in the Mataf area(near the Kaaba wall). The average walkingspeed is 0.3267 m/s.
From Figures 14 and 15 we see that the edgeof the Mataf moves faster than the center, thisphenomenon being known as the Edge Effect. The Edge Effect occurs when the edges of acrowd move faster than the center of the crowd.The density becomes higher and higher as onemoves from the edge of the Mataf towards thecenter. This phenomenon is explained by thefact that all pilgrims want to be near the Kaabawall. As a result, we find the density near theKaaba to be the maximum density. This datacan be used in validating of simulation tools.The mean walking speed for a group of pedes-trians moving in the pilgrim stream around theKaaba was 1.0816 m/s at the edge of the Matafand it was 0.3267 m/s for the same pedestri-ans groups moving inside the Mataf. Thesefindings agree well with the statistical resultsdiscussed in a previous section.
6. C omparison of walking speeds
One of the must-have results is to compare themean values and variances of walking speedsin both observations and simulation results. Adistinction will be made for walking speedsinside and outside of the Mataf platforms. Wemade a comparison between our plots derivedfrom the video observation and the fundamen-tal diagrams of (cf. fig. 16): • Walking speeds: – On the edge of the Mataf (free flowspeed) where the pedestrian densityis lower than 3 persons/m . – On the center of the Mataf. – On the Mataf inside near theKaaba wall where the pedestriandensity attains extreme levels (8-9persons/m ). 15unning title TrackingIndividualTargetsinHighDensityCrowdScenes
July 2014
AdvancesinComplexSystems
Vol. XXI, No. 1
Figure 16:
The Velocity-Density-Diagrams half-hour after Mid-Day Prayer (t =thalf-hour after Mid-Day). Average ofthe local speeds (cid:126) v ( (cid:126) r , t ) as a function ofthe local density ρ ( (cid:126) r , t ) . Our own dataare shown as red points. The blue pointscorrespond to the data obtained by (PM)[1]. The difference in velocity at lowerdensities can be explained by the fitness levelof pedestrians. All well-known fundamental diagrams pre-dict the same behaviour and have the sameproperties: speed decreases with increasingdensity. So the discussion above indicatesthere are many possible reasons and causesfor the speed reduction. For example there isa linear relationship between speed and theinverse of the density for pedestrians movingin a straight way [21]. However the pedes-trian walking speed can be affected by internaland external factors (such as the amount ofpedestrian inflow and outflow as well as theconfiguration of the infrastructure) not to for-get the physiology of the human body. It isfound that individuals walk faster in outdoorfacilities than in corridors [22]. According toPredtechenskii and Milinskii (PM) the averagewalking speed depends on the the walking fa-cility [1]. In other circumstances Weidmannconfirmed a linear relationship between thestep size length of walking pedestrians and theinverse of the density [23]. The small step sizemeans low pedestrian velocity, caused by re-duction of the available space with increasingdensity. The discussion above shows that thereare many possible factors influencing the fun-damental diagram. To identify these factors, it is necessary to exclude as many influencesof measurement methodology and short rangefluctuations from the data. Figure 16 showsthe average local speed (cid:126) v ( (cid:126) r , t ) as a function ofthe local density ρ ( (cid:126) r , t ) half-hour after Mid-DayPrayer (t = t half-hour after Mid-Day ). Our owndata is shown as red points. The blue pointscorrespond to the Milinski fundamental dia-gram. Moreover investigation data analysingthe Mataf area represented by blue points infigure 16 and showed that a reduction of theavailable navigation space illustrates the causesresponsible for the speed reduction with den-sity in pedestrian movement. The small de-viation in pedestrian walking speed at lowerdensity can be explained by the fitness level ofthe pedestrian.
7. M ovement R ecognition In the literature, there is a large number ofapproaches on detection and tracking of mov-ing objects from video images. Spatio-temporalanalysis has, in the past, been used to recognizewalking persons, where subspaces in the videoare treated as spatio-temporal volumes [24].Application of a Fourier transform to this datacan then identify data relating to movementacross the volume. This approach allowedpedestrian trajectories to be reconstructed fromvideo with high precision, taking advantagefrom the methods and the high developed com-putational technology. The common approachto detect movement is to produce comparisonimages (an image representing the different de-tails between two images) since this is compu-tationally efficient [25]. These comparison im-ages can then be computed further to estimatemovement vectors that describe the motion ofdrop-shaped objects captured in the respectiveimages. Murakami and Wada demonstrate an-other method, filing the difference frame, andinstead compare the properties of drops iden-tified in consecutive frames [26]. A drop thatis close to the position of a drop in a previousframe, and shares similar dimensions, is likelyto refer to the same figure. Motion vectors arealso used to find drop segmentation, which are16unning title
TrackingIndividualTargetsinHighDensityCrowdScenes
July 2014
AdvancesinComplexSystems
Vol. XXI, No. 1subsequently merged or separated for the pur-pose of analysis. The same approach is appliedto a 2D image to determine movement in 3Dspace. Extrapolating the movement of pedes-trians in 3D space from a 2D image allows fora far greater understanding of the interactionsbetween entities, but does require exceptionalcalibrations of equipment for complete accu-racy. The Murakami and Wada approach canbe used to analyse low-quality video streamsdue to the frame-differencing algorithm andsome trigonometry. Determining 3D motiondoes require precise knowledge of the angleand position of the camera, in addition to thebasic topology of the scene being analysed. But2D paths are easy to identify without these de-tails, (see fig. 17).
Figure 17:
Pilgrims paths. With a new computer algo-rithm developed during this research, the tra-jectories or movements of pedestrians acrossthe infrastructure over time are determined.Microscopic pedestrian fields require largeamounts of trajectory data of individualpedestrians. Every red solid curve corre-sponds to one pedestrian trajectory. The os-cillation in the pilgrims paths results fromthe huge pedestrian forces acting on everyindividual in the crowd.
In figure 17 we show the path of individu-als within the crowd. One clearly recognizesthat the movement around the Kaaba is not acircle movement. The tracking of a single in-dividual in the pilgrim stream indicates someoscillation movement around the main pathof the individual. It is caused by the physical repulsive and attractive forces acting on theindividual. Physical forces become importantwhen an individual comes into physical con-tact with another individual/obstacle. When alocal density of 6 persons per square meter isexceeded, free movement is impeded and localflow decreases, causing the outflow to dropsignificantly below the inflow. This causes ahigher and higher compression in the crowd,until the local densities become critical in spe-cific places on the Mataf platform.
8. A nalysis of the P ilgrimsmovement on the M ataf In the Mataf everything is dense and we havea compact state. The pilgrims have body con-tact in all directions and no influence on theirmovement; they float in the stream. This formsstructures and turbulences in the flow. Theseturbulences can be well observed in our videorecording. Density and velocity can also beseen. These observed Hajj rituals, especiallythe Mataf, showed some critical points in themotion of the pilgrims that we had not paidmuch attention to before. For example: theedge effect, density effect, shock-wave effectetc., and phenomena like these influence therestraint of the motion and are very importantto be considered.Our video analysis shows that the pedes-trian density decreases with the distance fromthe Kaaba wall, cf. figures 6, 7, and 8. It isthe same as the real behaviour of pilgrims onthe Mataf ritual (all pilgrims want to be nearto the Kaaba wall). Our video analysis aboutthe Mataf area indicates that, even at extremedensities, the average local speeds and flowsstay limited. This extremely high local densitycauses forward and backward moving shock-waves, which could be clearly observed in ourvideo. We can see a kind of oscillation on thepilgrims paths around the Kaaba, this oscilla-tion is caused by shock-waves and is affectedby the repulsive forces between the pedestriansin high density crowds (see fig. 17). 17unning title
TrackingIndividualTargetsinHighDensityCrowdScenes
July 2014
AdvancesinComplexSystems
Vol. XXI, No. 1
9. C onclusion and possibleimprovement
One of the significant challenges in the plan-ning, design and management of public facil-ities subject to high density crowd dynamicsand pedestrian traffic are the shortcoming inthe empirical data. The collected data concern-ing crowd behaviour using different techniques(image processing) and analysis of ordered im-age sequences obtained from video recordingis increasingly desirable in the design of fa-cilities and long-term site management. Wehave investigated the efficiency of a number oftechniques developed for crowd density esti-mation, movement estimation, critical placesand events detection using image processing.In the above sections and within this investi-gation we have presented techniques for back-ground generation and calibration to improvethe previously developed simulation model.Even though extracting information abouthuman characteristics from video recordingmay still be in its infancy, it is important tomention that the field of human motion analy-sis is large and has a history traced back to thework of Hoffman and Flinchbaugh [27]. In thefield of pedestrian detection techniques, more-over in the big area of computer vision, manyproblems have accumulated. In the humanmotion analysis, and also in the problem ofthe detection of moving objects, remain otherproblems, namely to recognize, categorize, oranalyse the long-term pattern of motion. Theinspection of the literature in the last decadeindicates increasing interest in event detection,video tracking, object recognition, because ofthe clear application of these technologies toproblems in surveillance. Recently many meth-ods have been developed to extract informationabout moving object like speed and density.Almost all these systems require complex inter-mediate processes, such as reference points onthe tracked objects or the image segmentation.One limitation of this current system is that thedetection failures for these intermediates willlead to failure for the entire system.Improvement of an algorithm to be able to reproduce traffic flow and to help in themicroscopic pedestrian data collection is veryessential. Moreover the automatic video datacollection will highly enhance the achievementof a system for higher pedestrian traffic densi-ties.
10. A cknowledgements
I would like to express my sincerest thanksand gratitude to Prof. Dr. G. Wunner for acritical reading of the manuscript, for his im-portant comments and suggestions to improvethe manuscript. Many thanks to Dr. H. Cartar-ius for his support during writing this work. R eferences [1] W. M. Predtechensky and A. I. Milinski. Personenströme in Gebäuden . Staatsverlagder Deutschen Demokratischen Republik,Berlin, russ: 1969, germ: 1971.[2] R. L. Knoblauch, M. T. Pietrucha, andM. Nitzburg. Field studies of pedestrianwalking speed and start-up time.
Trans-portation Research Record 1538. Washington(DC): National Research Council, Transporta-tion Research Board , Dec:27–38, 1996.[3] J. J. Fruin, American Society of Mechan-ical Engineers, and American Societyof Mechanical Engineers. Standing Com-mittee on Transportation.
Designing forPedestrians: A Level of Service Concept . Univ.Microfilm, 1970.[4] C. O’Flaherty.
Transport Planning and Traf-fic Engineering . Engineering village. Taylor& Francis, 1996.[5] J. J. Fruin. Pedestrian planning and design.
Elevator World. , 1987.[6] U. Chattaraj, A. Seyfried, andP. Chakroborty.
Comparison of pedes-trian fundamental diagram across cultures .2009.18unning title
TrackingIndividualTargetsinHighDensityCrowdScenes
July 2014
AdvancesinComplexSystems
Vol. XXI, No. 1[7] A. J. Bulpitt and N. Sumpter. Learningspatio-temporal patterns for predicting ob-ject behaviour.
In BMVC. , 1982.[8] B. Heisele and C. Woehler. Motion-basedrecognition of pedestrians.
ProceedingsFourteenth International Conference on Pat-tern. , 2:1325–30, 1998.[9] A. Sato, H. Koike, A. Tomono, andM. Tsuchikawa. A moving-object extrac-tion method robust against illuminationlevel changes for a pedestrian countingsystem.
Proceedings International Sympo-sium on Computer Vision. , 42, Issue 3:563–568, 1995.[10] S. P. Hoogendoorn and W. Daamen.Pedestrian behavior at bottlenecks.
Trans-portation Science , 39 (2):147–159, 2005.[11] A. Johansson and D. Helbing. From crowddynamics to crowd safety: A video-basedanalysis.
Advances in Complex Systems , 4(4):497–527, 2008.[12] M. Boltes, A. Seyfried, B. Steffen, andA. Schadschneider. Automatic extrac-tion of pedestrian trajectories from videorecordings.
Pedestrian and Evacuation Dy-namics, Springer-Verlag Berlin Heidelberg ,pages 43–54, 2010.[13] C. Papageorgiou and T. Poggio. Trainablepedestrian detection.
Proceedings 1999 In-ternational Conference on Image Processing ,4:35–9, 1999.[14] A. N. Marana, L. F. Costa, R. A. Lotufo,and S. A. Velastin. On the efficacy of tex-ture analysis for crowd monitoring.
SIB-GRAPI’98 1998, Proceedings , pages 354–61,1998.[15] Z. Zhang and M. Li. Crowd density es-timation based on statistical analysis oflocal intra-crowd motions for public areasurveillance.
Optical Engineering , 51(4),047204, 2012.[16] V. Verona and A. Marana. Wavelet packetanalysis for crowd density estimation. in Proc. of the IASTED International Sympo-sium on Applied Informatics, Cancun, Mexico ,2001.[17] X. Li, L. Shen, and H. Li. Estimation ofcrowd density based on wavelet and sup-port vector machine.
Trans. Inst. Meas. Con-trol (London) , 28(3):299–308, 2006.[18] X. Wu. Crowd density estimation usingtexture analysis and learning. in Proc. ofIEEE Conf. on Robotics and Biometics, IEEE,Kunming, China , 2006.[19] G. Sen, L. Wei, and Y. H. Ping. Countingpeople in crowd open scene based on greylevel dependence matrix. in Proc. of Intl.Conf. on Information and Automation, IEEE,Canada , 2009.[20] D. Helbing and A. Johansson. The dy-namics of crowd disasters: An empiricalstudy. arXiv:physics/0701203v2[physics.soc-ph] , 2007.[21] A. Seyfried, B. Steffen, W. Klingsch, andM. Boltes. The fundamental diagram ofpedestrian movement revisited.
J. Stat.Mech. , page 10002, 2005.[22] W. H. K. Lam and C. Y. Cheung. Pedes-trian speed-flow relationships for walkingfacilities in hong-kong.
Journal of Trans-portation Engineering, ASCE , 126(4):343–349, 2000.[23] U. Weidmann. Transporttechnik der fuss-gänger.
Technical Report Schriftenreihe desIVT Nr. 90, Institut fÃijr Verkehrsplanung,Transporttechnik, Strassen- und Eisenbahn-bau, ETH Zürich, Zweite, ergänzte Auflage ,1993.[24] Y. Ricquebourg and P. Bouthemy. Real-time human figure control using trackedblobs.
IEEE Transactions on Pattern Analy-sis and Machine Intelligence , 22(8):797–808,2000.[25] O. Masoud and N. P. Papanikolopoulos.A novel method for tracking and count-ing pedestrians in real-time using a sin-19unning title
TrackingIndividualTargetsinHighDensityCrowdScenes
July 2014
AdvancesinComplexSystems
Vol. XXI, No. 1gle camera.
IEEE Transactions on VehicularTechnology , 50(5):1267–78, Sept. 2001.[26] S. Murakami and A. Wada. An automaticextraction and display method of walkingpersons’ trajectories.
Proceedings 15th Inter- national Conference on Pattern Recognition ,4:611–14, Sept. 2000.[27] D. D. Hoffman and B. E. Flinchbaugh. Theinterpretation of biological motion.