Augmented Informative Cooperative Perception
Pengyuan Zhou, Pranvera Kortoci, Yui-Pan Yau, Tristan Braud, Xiujun Wang, Benjamin Finley, Lik-Hang Lee, Sasu Tarkoma, Jussi Kangasharju, Pan Hui
AAugmented Informative Cooperative Perception
Pengyuan Zhou ∗ , Pranvera Kortoc¸i ∗ , Yui-Pan Yau † , Tristan Braud † , Xiujun Wang ‡ , Benjamin Finley ∗ ,Lik-Hang Lee § , Sasu Tarkoma ∗ , Jussi Kangasharju ∗ , Pan Hui ∗∗ University of Helsinki † The Hong Kong University of Science and Technology ‡ Anhui University of Technology § University of OuluEmails: { firstname.lastname } @helsinki.com, { arthur.yau, lhleeac } @connect.ust.hk, [email protected], [email protected] Abstract —Connected vehicles, whether equipped with ad-vanced driver-assistance systems or fully autonomous, are cur-rently constrained to visual information in their lines-of-sight. Acooperative perception system among vehicles increases their sit-uational awareness by extending their perception ranges. Existingsolutions imply significant network and computation load, as wellas high flow of not-always-relevant data received by vehicles. Toaddress such issues, and thus account for the inherently diverseinformativeness of the data, we present Augmented InformativeCooperative Perception (AICP) as the first fast-filtering systemwhich optimizes the informativeness of shared data at vehicles.AICP displays the filtered data to the drivers in augmented realityhead-up display.To this end, an informativeness maximization problem ispresented for vehicles to select a subset of data to display to theirdrivers. Specifically, we propose (i) a dedicated system designwith custom data structure and light-weight routing protocol forconvenient data encapsulation, fast interpretation and transmis-sion, and (ii) a comprehensive problem formulation and efficientfitness-based sorting algorithm to select the most valuable data todisplay at the application layer. We implement a proof-of-conceptprototype of AICP with a bandwidth-hungry, latency-constrainedreal-life augmented reality application. The prototype realizesthe informative-optimized cooperative perception with only 12.6milliseconds additional latency. Next, we test the networkingperformance of AICP at scale and show that AICP effectivelyfilter out less relevant packets and decreases the channel busytime.
I. I
NTRODUCTION
Connected and autonomous vehicles are closer than everto becoming a reality. Specifically, modern communicationtechnologies such as cellular vehicle-to-everything (C-V2X)and dedicated short-range communications (DSRC) facilitatelarge-scale vehicular communication thanks to significant im-provements in bandwidth, latency, and reliability. Additionally,novel regulations provide a beneficial legal context for theoperation of autonomous vehicles on public roads [1]. Thispaves the way for the deployment of applications that lever-age vehicular communication to provide more information tohuman and AI drivers, and thus improve road safety. Currently,autonomous vehicles and advanced driver-assistance systems(ADAS) rely heavily on on-board sensors to identify and eval-uate potential dangers and take necessary actions. However,most current solutions are limited to a single vehicle pointof view, sensing only the nearby objects within their line-of-sight. As such, the vehicle’s sensing capabilities are regularlyobstructed by other vehicles, and thus depriving the driver ofpotentially useful information. Leveraging current and future
CAR CAR CARPPL PPL PPLCAR CAR CARPPL PPL PPL
Fig. 1: Illustration of a na¨ıve cooperative perception system.The driver of a following vehicle is shown all the objectsdetected by a leading vehicle. In contrast, AICP filters theseobjects to only show the critical objects (e.g., the pedestriansin the pink bounding box) to avoid information overload.communication networks, the vehicle can aggregate the per-ception of multiple nearby vehicles, i.e., cooperative (collec-tive) perception [2], [3], and provide a driver (human or AI)with a holistic view of the road situation. This concept hasbeen adopted by the European Telecommunications StandardsInstitute (ETSI), which is working on Cooperative/CollectivePerception Service standardization [4], [5].Existing works focus on timely and synchronized informa-tion distribution, data fusion, or communication overhead [2],[3], [6], [7]. However, the informativeness of the sharedperception data has been largely overlooked. Cooperativeperception, and more generally safety applications that rely oncommunication among vehicles, require deployment at scaleto provide a holistic vision of the road. Such a pervasivedeployment leads to a significant strain in terms of network,computation resources, and driver awareness caused by theconstant information dissemination across a large number ofvehicles. However, only a part of the disseminated informationis of interest to the drivers.Figure 1 shows an example of a following vehicle’s visionin a leading-following vehicle scenario with na¨ıve cooperativeperception with augmented reality (AR). The leading vehiclecaptures the objects within its line-of-sight and broadcastsinformation about the detected objects. The following vehiclecalculates the position transformations and renders all theobjects, shown in green boxes, from the received messages.Extending such a system to city-block-level perception withno information filtering leads to a massive number of objectsbeing displayed to the drivers. This overwhelms their vision a r X i v : . [ c s . MM ] J a n nd thus negatively impacts their driving experience. In fact,a driver’s decision time increases logarithmically with thenumber of stimulus or objects [8]. Additionally, limiting thenumber of objects to the human cognition capacity of about7 ± (1) We propose a system design for AICP. The design includesa dedicated data structure, the vehicular data unit (VDU),designed for informativeness-focused information filtering andtransmission. We also describe the full-stack networking pro-tocol that the system employs to utilize VDU. (2)
We formulate the informativeness problem in coopera-tive perception systems from the object , message , and vehi-cle level , and propose a weighted fitness sorting algorithmbased on Mahalanobis distance for fast, yet comprehensiveinformativeness-based filtering. The algorithm provides filter-ing at the application level to display only the most importantinformation shared by nearby vehicles, and thus preventingdrivers from information overload. (3) We implement a prototype of the proposal using a co-operative perception application on an Augmented RealityHead-up Display (ARHUD). Next, we evaluate the networkingperformance of AICP at scale with simulations.We note that AICP is network-agnostic and does not dependon any particular feature of the underlying network, as suchthe system can be seamlessly integrated into current and futurecommunication systems such as C-V2X and DSRC.The rest of the paper is structured as follows. Section IIdiscusses related works and states the key motivations be-hind AICP. Section III details the system architecture, datastructure, and routing protocol. Section IV models the sys-tem and formulates the problem. Sections V describes theweighted fitness sorting algorithm to calibrate the assessmentof informativeness. Section VI shows the proof-of-conceptimplementation of AICP and its performance in different sce-narios. Section VII presents the simulation setup and results.Finally, Section VIII discusses system limitations and potentialsolutions, and Section IX concludes the work. II. R
ELATED W ORK
As this work touches on different research areas includingcooperative perception, information filtering, and AR in thecontext of vehicular networks, the related work also falls intothese areas. In terms of general vehicular cooperative per-ception, Kim et al. proposed a framework addressing severalimportant problems in the field such as map merging, commu-nication uncertainty, and sensor multi-modality [11]. Further-more, [12] proposed and analyzed different message formatsbased on ETSI ITS 5G [13] to exchange local sensory dataamong road participants for collective perception. Thereafter,they proposed a multimodal cooperative perception systemwith a focus on engineering feasibility [3], and generalizedthe work with a mirror neuron inspired intention awarenessalgorithm for cooperative autonomous driving [14].The critical area of filtering mechanisms in vehicular co-operative perception is still very new and has only a fewkey works. Garlichs et al. [7] in 2019 suggested a set ofgeneration rules to reduce the transmission load while guaran-teeing perception capabilities. This proposal was later addedto the ETSI standard [5]. Thandavarayan et al. studied theETSI standards and conducted an in-depth evaluation of themessage generation rules [10]. They investigated the trade-off between the perception capabilities and communicationperformance under current standards and concluded that fur-ther optimization is needed to reduce information redundancy.To this end, Aoki et al. [15] applied a deep reinforcementlearning approach to reduce the information sent betweenvehicles by only forwarding information about objects that arenot likely to have been seen directly by surrounding vehiclesthemselves. Finally, works in the area of ARHUD, an in-cardeployment of AR that visualizes information in the driver’sline-of-sight, are also related. For instance, [6], [16] explorehow to share augmented vision between two vehicles. Otherstudies consider connecting multiple mobile points of view torecompose a scene in 2D or 3D [17]. However, these studiesmainly focus on image stitching, and thus overlooking aspectssuch as that of data redundancy.To the best of our knowledge, cooperative augmented ve-hicular vision at scale still requires additional research onefficient information filtering. Specifically, we believe that AR-powered cooperative perception needs a comprehensive solu-tion to maximize the informativeness of the data shared amongvehicles to improve the driving experience while increasingroad safety. Therefore, in this work we propose AICP, asystem that lessens the burden on the network through efficientdata filtering. Consequently, only the most relevant data isbroadcast to vehicles, which in turn sort such data to maximizethe informativeness that they yield to the driver.Overall, we find that AR-powered cooperative perceptionneeds a comprehensive solution to maximize the informa-tiveness for better driver experience. In this paper, we pro-pose AICP with a detailed system modelling, protocol designand algorithms. ameras IMUObjects DataCommunicationInterface ARHUD MCUWeighted FiltrationCommunicationInterfaceCMRVDUSender ReceiverVDUCMR
Relay?
REL/ABS ABS/REL
Fig. 2: System components: inertial measurement unit (IMU),microprocessor control unit (MCU), relative values (REL),absolute values (ABS), vehicular data unit (VDU), and con-textual multihop routing (CMR) protocol.III. S
YSTEM O VERVIEW
This section describes the proposed system in terms of themajor components, data structure, and the routing protocol.
A. System Architecture
We consider a connected system of vehicles equipped withsensors and wireless communication modules that can collectand share sensory data with each other. We assume the systemhas capabilities including accurate positioning and localiza-tion [18], [19], relative velocity estimation, distance and angleestimation [20], and perspective transformation [6]. In thiswork we skip the details of the aforementioned techniques andfocus on the aspect of information filtering. Figure 2 depictsthe system architecture including the major data flows betweenkey components. These data flows include: • IMU sensors in each sender collect sensory data. Eachsender detects the objects captured by the on-board camerasand corresponding information such as distance, relative ve-locity and moving direction of the objects. • The sender system transforms the object data from relativevalues to absolute values based on IMU data. For instance, thesystem transforms the relative velocity of a detected object toabsolute velocity by adding its own velocity. • The data are then encapsulated into VDUs (see Sec. III-B). • The sender system encapsulates networking layer informa-tion into VDUs according to the CMR (Sec. III-C). • The senders and receivers exchange data packets via wire-less communication interfaces. • Each receiver decides whether to forward the receivedpacket based on a network layer filter (see Section III-C). • Each receiver transforms the absolute values of the objectsto relative values based on their own IMU data. • Each receiver filters the received VDU based on a filteringalgorithm (see Algorithm 1).
GPSSecurityServices Message Set DictionaryUDPIPV6 CMRPHYMACPDCP/RLC or LLCVDU: IMU OBJTYPE TIMEDIR TTL
Fig. 3: AICP protocol stack. • Each receiver ARHUD renders the filtered informationthrough AR to enhance the driver’s situational awareness. TheMCU perform maneuvers based on the filtered information.As such, each vehicle is able to display the most importantsituational information in real-time to facilitate safe driving.Next, we describe the VDU and CMR in detail.
B. Vehicular Data Unit (VDU)
Figure 3 depicts the overall protocol stack deployedby AICP. AICP is network-agnostic and can be deployed ontop of any V2V broadcast-style protocol such as DSRC. CMRis deployed as the routing protocol to provide context-awarerouting in a broadcast network environment, in parallel tothe traditional UDP/IP stack. Finally, the VDU contains theinformation required at the application layer. To acceleratedata processing during filtering, we propose the vehicular dataunit (VDU), a language- and platform-neutral data structurefor vehicular information encapsulation. A VDU is comprisedof multiple metadata fields, each of which is a key-value pairor key-value map. The metadata fields include: • TYPE – whether the message is a safety or non-safetyapplication message (pair). • TIME – the time the information was first created (pair) . • IMU – the information of IMU sensors (map), i.e., { GPScoordinates , Velocity , diRection , Category } . • OBJ – the information of detected objects (map).Similar to the SAE J2735 standard [21], the protocol stackdefines a message set dictionary to specify the VDU struc-ture and provides sufficient background information to allowvehicle systems to properly interpret the message. Togetherwith the segmented data blocks, the dictionary extends thesystem compatibility by allowing different VDU structures.The dictionary and VDU also speed up the look-up processof fields such as
TIME which can help to determine whetherthe received information is outdated (see Section IV).
C. Contextual Multihop Routing (CMR)
Vehicle-to-Vehicle (V2V) communication suffers from ashort communication range due to signal attenuation causedby obstacles such as nearby vehicles and buildings [22]. Assuch, we assume that only entities within the line-of-sight We employ elapsed time since Unix epoch to record timestamps. f the transmitter can receive the transmissions. Therefore,packet forwarding is crucial to extend the range over whichinformation can be propagated. To date, most DSRC and C-V2X standards assume data transmission over a broadcastmechanism. Broadcast transmission allows a vehicle to effi-ciently forward information to other vehicles in their imme-diate vicinity. However, broadcast transmission suffers frommultiple drawbacks in multihop communication. Unregulatedbroadcast transmission results in significant data redundancythat affects the system at every level, from increased loadand congestion on the transmission medium to large amountsof unnecessary information being forwarded to the drivers.To address these concerns, we introduce CMR as the routingprotocol for packet forwarding.CMR enables the following features. (i)
Directional routingin a multihop broadcast transmission.
We consider that anobject detected by a vehicle is relevant for all immediateneighbors (accessible within a single hop). However, theinformation is only relevant to vehicles further away fromthe source vehicle if the vehicles are going in the samegeneral direction as the source. As such, vehicles only forwardpackets to other vehicles where this is the case. (ii)
Hoplimit for geographic relevance of information.
After a certaingeographic distance, the information also loses its relevance.As the number of hops acts as a rough proxy for geographicdistance, a hop limit thus prevents further propagation ofirrelevant information. (iii)
Lightweight message filtering.
Wedesign the routing mechanisms to rely on only a few atomicoperations in order to minimize the packet forwarding time.To provide these functions, CMR in this work relies on twodifferent metrics, namely
GPS and
TTL , respectively referringto the coordinates of the detected object and a counter usedto enforce the hop limit (see Figure 3). The GPS coordinatesof the detected object are encoded over two fields of bitsto achieve precision of at least a meter. Directional routing . We embed the heading direction andthe location of the source vehicle in the packets (field
DIR and
GPS , respectively) to provide directional routing. The on-board magnetometer returns a -bit value between and °, similar to a compass. Each vehicle receiving the messagecomputes its own direction and compares it to the DIR and
GPS fields of the message. If the receiver vehicle is headingtowards the source vehicle or if both vehicles are heading inthe same direction, then the packet is forwarded; otherwise,the packet is dropped. We also propose a directional antennapattern that helps support this directional routing approach(refer to Section VI for more details).
Hop limit . Upon reception, each receiver decreases the valueof
TTL by . When the TTL reaches , the packet is dropped.The hop limit is defined by the original transmitter andmay depend on many factors, including vehicle density, roadconfiguration (type or geography), and the vehicle’s speed.For instance, the hop limit may be higher on a crowded urbanintersection with a large number of vehicles and pedestriansspread in multiple directions than on a highway during off- TABLE I: Summary of used notations. Symbol Definition N Set of all vehicles considered in the system V t Set of velocities V t = { v t , . . . , v tN } of N vehicles TTL th Initial TTL (time-to-live) of detected objects in the system ϑ O Informativeness of object Ot c Time at which a message is created I i ( t ) Informativeness of message i at time tr Decay rate of informativeness I over time x j,i,o ( t ) Binary variable indicating if object o contained in message i is received by vehicle j at time t peak hours. In the rest of this work, we consider the hop limitto be fixed for the sake of simplicity. Lightweight message filtering . CMR provides an initialfiltering at the routing stage. However, as a routing protocol,priority should also be put on simplicity for performancereasons. As such, we design the protocol to include only alimited number of atomic operations, allowing for very fastforwarding decisions. According to the policy of CMR, eachreceiver drops a received message if the
DIR difference isbigger than the threshold or the value of
TTL is . Otherwise,the receiver passes the packet to the upper layers for furtherdecapsulation while forwarding the packet via CMR broad-cast. As such, CMR improves transmission efficiency by notforwarding likely irrelevant messages to the receiving vehicles. Note that unlike common routing algorithms such as DV-CAST [23], Greedy Perimeter Stateless Routing (GPSR) [24]and its variants such as [25], [26], CMR focuses on fil-tering low-informativeness packets instead of improving thecommunication efficiency. Therefore it is a complementaryprotocol to the efficiency-focused routing protocols, insteadof a replacement.IV. S
YSTEM M ODEL AND P ROBLEM F ORMULATION
This section details the system model and the problem ofmaximizing the informativeness of the displayed objects.
A. System Model
The system includes the set N = { , , ..., N } of N vehicles driving in the considered area, with velocities V t = { v t , v t , ..., v tN } at time t , respectively. The mobility of thevehicles is exogenous to the system. Each vehicle is equippedwith an ADAS or autonomous system consisting of severalcameras facing varying directions for comprehensive visionaround the vehicle (radar, lidar and ultrasonic sensors are op-tional and not a mandatory requirement of AICP), GNSS/IMUfor real-time kinematic and positioning, and wireless interfaces(DSRC or C-V2X) for communications with other deviceson the road. Messages are sent with a frequency between and Hz and the message size is limited to
Bytes, asspecified in the C-V2X standard [13], [27]. The encapsulationand decapsulation of the messages follow the protocol standarddefined in Figure 3. We model the data propagation fromthree parallel levels, namely the object level , message level ,and vehicle level . AICP decides whether to display an objectased on the object level informativeness, to forward a packetbased on message level informativeness, and targets optimizingperformance based on vehicle level informativeness.
Object level . The processing result of each image framecontains a list of detected objects, each of which is definedas O (cid:44) { D, V, R, C } , where D, V, R, C denote the
Distance,relative Velocity, diRection and Category of the object, re-spectively. The rationale of the choice of these parameters isjustified by the fact that an object O has a higher chanceof causing an accident if (i) it is close to the vehicle, (ii) isgetting closer to the vehicle, e.g., catching up with the vehiclefrom behind or coming right at the vehicle, and (iii) is on theheading direction of the vehicle. Additionally, the rationalefor having an object category relies on the fact that certainobjects could cause or sustain greater injury in an accident;e.g., a pedestrian should raise more attention than a parkedvehicle or a trash bin on the side of the street.These parameters are intertwined with each other, andsuch dynamics are crucial to determine a model upon whichwe define the informativeness of an object. For instance,mutual time and space relationship is, in fact, at the basis ofmodern methodologies to evaluate accidents by analyzing thecollision area [28], [29]. An examination of reported accidentsinvolving autonomous vehicles in California showed that mostaccidents occur at cross sections in suburban roads, with mostaccidents reporting rear or front damage [30]. These findingsindicate that the direction with which vehicles move greatlyaffects the probability of an accident, especially if the vehiclesare at a short distance from each other. Furthermore, we needto factor in a higher informativeness for objects that fall intothe people category, for instance.Upon such considerations, we express the informativeness ϑ of an object O as: ϑ O = (cid:0) ( αD + βV ) γR (cid:1) ωC (1)where α , β , γ , and ω are weighting parameters of the fourpriority attributes. Note that to improve the comprehension ofthe informativeness, we present a weighted fitness sorting algo-rithm (Section V-B) to calculate the inter-weight relationshipsusing Mahalanobis distance matrix [31]. Learned from labelleddatasets, the matrix realizes informativeness categorizationwith negligible delay.More complicated computations such as machine learningalong with accident reports of autonomous vehicles [32]can incorporate the understanding of a chain of scenarios;however, they might suffer from additional costs. We leavethe investigation of such an approach as possible future work. Message level . The messages received by each vehicle mayarrive at varying times and with varying delays. In fact, dueto the highly time-sensitive nature of the warning messagesfor assisted-driving, vehicles need to establish the timelinessof the message [33]. To this end, the system extracts from the VDU the time at which a given message was created (see TIME in Figure 3), here denoted as t c , and uses it to evaluatethe timeliness. For a message i , its informativeness can becalculated as I i ( t ) = (cid:18) ϑ i (cid:16) TTL i ( t ) TTL th (1 − r ) (cid:17)(cid:19) ( t − t ic ) (2)where ϑ i = (cid:80) o ∈ O i ϑ o denotes the informativeness of message i and O i denotes the detected objects contained in message i , r is the rate at which the informativeness of the messagedecays over time, and t denotes the current time. The decayrate r is strictly connected to the time limit within which suchmessages are considered up-to-date and relevant . In fact, r isa system parameter that can be tuned for specific conditions.For instance, a faster decaying rate is required when conditionschange quickly, such as driving on the highway at high speeds.While we need to relax the decay rate r for conditions withslower speeds (e.g., city center). For instance, a decay rateof r = 0 . halves the informativeness of a message inabout seconds. TTL th characterizes the hop limit definedby CMR (see Section III-C). TTL i ( t ) expresses the remainingtime-to-live, i.e., the number of hops a message can still beforwarded, of message i at time t . Vehicle level . At the vehicle level, a vehicle can (i) evaluatea received message as informative and display some objectsfrom that message to the driver and subsequently broadcastthe message, (ii) evaluate a received message as irrelevant tothemselves but still broadcast the message, or (iii) drop thereceived message if its information is outdated, and thus notrelevant to any vehicle in the network.Upon receiving a multitude of messages, a vehicle derivesthe informativeness of the objects within. We assume a vehicle n ∈ N receives M messages at time t from other vehicles inthe network, and express their informativeness as below. I n ( t ) = |N | (cid:88) j =1 j (cid:54) = n M (cid:88) i =1 (cid:88) o ∈ O i (cid:18) ϑ j,i,o (cid:16) TTL i,o ( t ) TTL th (1 − r ) (cid:17)(cid:19) ( t − t i,oc ) x j,i,o (3)The binary variable x j,i,o is if object o ∈ O i containedin message i is received by vehicle j ( x j,i,o = 1 ), and otherwise.The number of received messages (and objects) can increasedrastically when a vehicle drives into dense traffic. Displayinga large number of objects to the driver is unwise as thisoverloads their vision. In fact, a vehicle must first filter theincoming messages and select only those with the highestinformativeness to mitigate such an issue. To this end, we nextpresent a solution based on optimization of informativeness. The timeliness with which we receive given messages as well as the datacontained within is strictly related to the capability to identify potential harm-causing objects. We incorporate these notions into the term informativeness and use it throughout the rest of the article for the sake of conciseness. . Problem Formulation
The following formulates an optimization problem for a ve-hicle to select objects whose informativeness helps to identifyimminent risks, and thus increase road safety.Specifically, the problem is defined as below. max j,i,o I n ( t ) (4)Subject to: |N | (cid:88) j =1 j (cid:54) = n M (cid:88) i =1 (cid:88) o ∈ O i x j,i,o ≤ L, ∀ n ∈ N (5) TTL i,o ( t ) > , ∀ i (6) x i,j,o ∈ { , } , (7) ∀ j ∈ [1 , , ..., |N | ] , ∀ i ∈ [1 , , ..., M ] , o ∈ O i The objective function in Eq. (4), defined as the summationof the informativeness of the objects contained in the messagesincoming from other vehicles, expresses the collective infor-mativeness that vehicle n conveys at time t . Eq. (5) limits thenumber of selected objects used to convey information to thedriver. This allows us to display only clear and limited audioand visual content to a driver [34], [35]. Eq. (6) specifies thatthe time-to-live of an object must be, trivially, larger than .Finally, Eq. (7) restricts the x j,i,o variable to binary integervalues. Given a suitable value of L , AICP selects L objectswith the largest informativeness to display to the driver.We also note that we assume that object resolution (alsoknown as entity resolution) is performed before this infor-mativeness selection step. Object resolution is the process ofrecognizing that multiple perceived or received objects areactually the same object just seen by multiple vehicles fromdifferent angles. As this resolution process is a well-knowntopic in itself we do not focus further on the process but insteadrefer to [36] which summarizes different object resolutionmethods. These methods typically have super-linear but sub-quadratic time complexity with the number of objects andthus should not significantly impact the system performance inpractice. We look to study resolution techniques empirically infuture work. Next we present the details of the fitness sortingalgorithm in Section V, which allows such a selection with atime complexity of O ( M ) .V. P RIORITIZED S ORTING A LGORITHM
In this section we propose the details of the filtering andprioritized sorting algorithm at the application layer. Thealgorithm finds the relationships between the attributes thatdefine the informativeness to provide a more comprehensiveunderstanding than the linear weights in Eq. (1).
A. Warm-up Radix Sorting
We present a warm up solution that orders the list O v ofobjects recently received by a vehicle v . The radix sortingalgorithm, represented by Sort () , arranges the orders of the tuples in O v . Without loss of generality, the algorithm Sort ( a ) orders the tuples in O v in ascending order in the attribute a .First, we assume the order Distance (D) > I relative Velocity(V) > I diRection (R) > I Category (C) of the four attributes ina tuple. Second, we sort the tuples in O v by running Sort ( C ) , Sort ( R ) , Sort ( V ) , and Sort ( D ) sequentially. That is, we startsorting from the least significant attribute and move up tothe most significant one. We observe that the running timeof this solution is about O (4 n ) since the running time ofthe radix sorting algorithm Sort () is O ( n ) and we need toprocess four attributes. However, a drawback to this solutionis the assumption of a monotonic relationship between thefour attributes, i.e., attribute a is more important than another b , regardless of the actual value of b . B. Weighted Fitness Sorting
In the following we introduce a more advanced method,which not only considers the impact of the values of the fourattributes, but also has a faster running time compared to theradix algorithm. To do so we first must assume that we havea labeled dataset D in which a large number of tuples P (cid:63)i , i = 1 , , ..., N are collected from vehicles and then labeledas either Requires Attention or Does Not Require Attentionby human experts on traffic safety analysis. If L (cid:63)i denotes thelabel of tuple P (cid:63)i , the labeled dataset D can then be representedas D = { ( P (cid:63)i , L (cid:63)i ) , i = 1 , , .., N } . For ease of illustration, weassume L (cid:63)i ∈ { , } , where represents Does Not RequireAttention and represents Requires Attention. We note thatthe dataset D could be obtained by using the large volumes oftraffic data that connected cars stream back to network centersfor data analysis [37], [38]. Furthermore, there are numeroussemi-supervised classification algorithms [39], [40] that canbe used to build a large dataset D from a small initial labeleddataset which can be generated manually by traffic experts oreven automakers.Next, we discuss the details of the weighted sorting algo-rithm. Without loss of generality, we assume that the fourattributes D , V , R and C are represented by -th, -th, -th and -th attributes, respectively. Given a tuple P = ( D, V, R, C ) represented by P = ( x , x , x , x ) , then a labeled tuplein dataset D is represented by ( x (cid:63) , x (cid:63) , x (cid:63) , x (cid:63) , L (cid:63)i ) . To betterdefine the relationship between the weights in Eq. (1), wedefine a filter F which weights the four attributes and theirrelationship by a × matrix M shown as follows M = m , m , m , m , m , m , m , m , m , m , m , m , m , m , m , m , , (8)where m i,j > ( i, j ∈ { , , , } ) is the weight assignedto the relation between the i -th and j -th attributes. Given atuple P = ( x , x , x , x ) , filter F shall compute a fitness(Mahalanobis distance) as follows: F ( P ) = P × M × P (cid:48) (9) lgorithm 1: AICP full-stack filtering algorithm
Networking layer filteringthread
CMR : Determine drop or forward&process
PKT according to the CMR protocol (see Sec. III-C);
Application layer filteringthread
Fitness calculation : Update the fitness matrix M (see Eq. (8)) withnew datasets according to Eq. (11), offline; thread Weighted sorting : Calculate the fitness values of the objects inreceived messages according to Eq. (9); Sort the fitness values; thread Display : Display the first L objects in the sorted queue toachieve Eq. (4) under the constraint of Eq. (5); = [ x x x x ] m , m , m , m , m , m , m , m , m , m , m , m , m , m , m , m , x x x x For ease of illustration, we use d ( P i , P j ) to represent thedifference of the fitness between two tuples P i and P j , i.e., d ( P i , P j ) = ( P i − P j ) × M × ( P i − P j ) (cid:48) (10)We find suitable values for the matrix M used by filter F by leveraging the knowledge from the labeled dataset D . Let D represent a subset of D where each tuple is labeled with , and D = D − D . The idea is to find a matrix M (cid:63) thatmaximizes the difference of the fitness between the subset D and D , i.e., the fitness of the tuples in D shall be as smallas possible, while that of the tuples in D can be as large aspossible or vice-versa. More formally, we solve the followingoptimization problem. arg max M (cid:88) i (cid:54) = j,L (cid:63)i (cid:54) = L (cid:63)j d ( P (cid:63)i , P (cid:63)j ) − (cid:88) i (cid:54) = j,L (cid:63)i = L (cid:63)j d ( P (cid:63)i , P (cid:63)j ) (11)The formula on the left side of the subtraction sign definesthe distance between two tuples with different labels (onetuple is labeled and the other ), while the formula on theright side defines the distance between two tuples with thesame label (the tuples are labeled either or ). This is aclassic metric learning problem for which numerous numericaloptimization algorithms are available [41]. This analysis canbe carried out offline, and the obtained M matrix can be usedto quickly evaluate whether a newly received tuple P needsfurther processing by calculating its fitness as shown in Eq. (9).Next, we analyze the time complexity of the weightedsorting solution. As mentioned, obtaining M offline allowsus to neglect its computational cost. Upon obtaining M , the CamerasOBJ DetectionOBJ TrackingVelocity EST Distance ESTOBJ Info ARHUDWeighted FiltrationOBJ InfoREL/ABS ABS/REL
Sender Receiver
Fig. 4: Data flow of the POC prototype system.cost is O (1) to compute the fitness value of a tuple, andthus O ( N ) to calculate the fitness values of the N tuplesin the O v list. In addition, the cost is O ( N ) to sort thefitness values using the radix sorting algorithm. Hence, thetotal computational cost is O (2 N ) , which is less than thatof the warm-up solution (which is O (4 N ) ). Furthermore, theweighted solution is advantageous as it takes into accountthe values in each attribute and their relations. As such,we summarize the full-stack filtering algorithm of AICP inAlgorithm 1. VI. E VALUATION
A. Implementation
Following the system design in Section III, we imple-ment a proof-of-concept (POC) prototype of AICP as asender/receiver system. The sender detects objects in the line-of-sight and shares them with nearby vehicles in real-time.The receiver filters the received objects through the sortingalgorithm presented in Section V-A and renders them to thedriver in ARHUD. As shown in Figure 4, the sender consistsof five components: (i) object detection, (ii) object tracking,(iii) distance estimation, (iv) velocity estimation, and (v) rela-tive/absolute value transform; the receiver consists of (i) valuetransform, (ii) weighted filtration, and (iii) AR display.We use a video recorded by the front cameras of twovehicles driven across a European capital city center, onefollowing the other. The sender streams the video into Yolo-v5 Object Detector [20] and conducts object detection in real-time. The detection outputs tuples that consist of the positionsand the labels of the objects, as well as confidence scores.Next, the system feeds the results to the object tracking component, which maintains a list of the objects tracked inthe previous frames. First, such a component calculates theIntersection over Union (IoU), defined as area of overlaparea of union , betweenthe objects in the previous and the latest frames. Next, it usesa greedy algorithm (quicksort) to sort the similarity accordingto the IoU scores [42].The system passes the tracking result to the velocity esti-mation and distance estimation components simultaneously.The velocity estimation uses the historical positions of anobject to estimate the speed and direction of its movement. a) Parking lot. (b) Intersection. (c) Pedestrians walking across.
Fig. 5: View of the POC prototype system. At the top we see all the objects detected by a sender vehicle, whereas at thebottom a receiver vehicle sees only the objects that result from our filtering algorithm. For the sake of comprehensibility, weshow all the objects that the sender vehicle detects, although a vehicle does not need to render its own objects.The distance estimation uses the object’s position and the ratiobetween the object area and the average size of the object toestimate the distance to the object. We categorize the distanceinto three classes, namely nearby, middle, and far away.After gathering the objects’ positions, labels, velocities, anddistances, the sender broadcasts the messages. The receiverapplies the weighted fitness sorting algorithm to select the top L important objects out of the received message and rendersthem to the driver.We calculate the Mahalanobis distance in Eq. (9) usingthe well-known large margin nearest neighbor (LMNN) [43]due to its simplicity and fairly well performance. Overall, thevideo captures 18488 images with 30 FPS (frame per second).We set the maximum number of objects to be detected as25 objects per frame. The final number of detected objectsis 75876. To prove the performance of the weighted fitnesssorting algorithm (V-B), we first manually categorize theobjects into two label groups, “require attention” and “notrequire attention”, as described in section V-B. For simplicity,we first use a predefined policy to categorize the objects intolabelled datasets. We define an object as requiring attentionwhen it has a distance is less than 23 meters (safe breakingdistance when driving at a safe speed 30mph), or velocitylarger than 30, or is a pedestrian. As a result, we get 22996 and52880 objects for the two categories, respectively. We use the metric-learn library contributed by the authors of [56] toimplement LMNN for computing Mahalanobis distance matrix(see Eq. (8) to Eq. (10)). The matrix computation takes 225.79seconds per 10000 objects. As described in V-B, the matrixcomputation is offline thus does not impact real-time systemperformance. Using the learned matrix, each receiver can sort100 received objects within a millisecond on average. We opensource the code of the weighted fitness sorting algorithm andthe 75876-object data extracted from the video at [44]. B. Showcase Performance
Table II summarizes the latencies of the processes. Asshown, the overall latency in the sender system is only . TABLE II: Latency breakdown of the POC components.
Task Execution Time (ms)
Sender Pre-processing . Object Detection . Object Tracking . Velocity & Distance Est. . Receiver Sorting AICP Overall Latency . milliseconds and thus has a negligible impact on informationdissemination. The additional latency added to the receiver endis mainly the sort latency, which is ∼ millisecond on averageand does not affect the networking performance. As shown inFigure 5, AICP effectively improves the understandability ofthe cooperative perception system by pruning shared percep-tion information and showing only the most critical objects.In different scenarios, e.g., in a parking lot, at an intersection,and with pedestrians crossing the street, the filtered displaysare much easier to comprehend and thus provide better fa-cilitation to driving. In comparison, cooperative perceptionsystems without AICP would display numerous objects like inFigure 5a and Figure 5b or unimportant objects like the trafficlights in Figure 5c. Note that in different areas the system canimprove performance by adapting L (the number of selectedobjects defined in Eq. (5)). For instance, the number can besmaller in a parking lot where less objects are moving while atan intersection it may require to display more objects. Cloudservice like Google Map can be used for area identification.VII. S IMULATION
Following the performance of the prototype shown in Sec-tion VI-B, we next test the performance of AICP through alarger scale simulation. We open source the core scripts anddatasets of the simulation at [44].ABLE III: Parameters
Simulation Parameter Value
Radio Propagation Model Two-Ray Interference Model [46]Shadowing Model Obstacle Shadowing Model [47]IEEE 802.11p Bit Rate MbpsTransmission Power mWNoise Floor − dBmAntenna Height . mAntenna Type Monopole [48], Front-Rear [44]Number of Vehicles 212 [44]Simulation Area x mSimulation Time s CMR Parameter Value
Beacon Generation Rate HzHop Limit Max Source Heading Direction Deviation °Max Source Distance mPacket Size Bytes
A. Simulation Setup
As pointed out in Section III-C, CMR focuses on filter-ing low-informativeness packets instead of communicationefficiency. To isolate the effect of CMR, we exclude anycommunication efficiency-focused routing protocols such asGPSR [24] and DV-CAST [23]. Instead, we compare thetransmission statistics with and without CMR in city-scaleV2V simulations. We select an area of size km by kmaround London city center and generate traffic utilizing a real-life dataset . Analogue models . We simulate the traffic during a peakperiod ( pm) using Veins [45], an open source frameworkfor running vehicular network simulations.
Veins is basedon
OMNeTpp , an event-based network simulator, and
SUMO , aroad traffic simulator. We run the simulation for seconds. Toensure realism we employ the two-ray interference model [46]for radio propagation. The model improves over the vanillatwo-ray ground model by also capturing the ground reflectioneffects. We also employ the obstacle shadowing model [47] tocapture the effects of buildings on signal transmissions. Theupper part of Table III details the parameters. Routing protocol . We use IEEE 802.11p as the base network-ing protocol for V2V communications. Following the design ofCMR, we set the hop limit of each message to , the maximumconcerned source distance to meters, and the maximumheading direction difference between the source vehicle andthe receiver to degrees. The C-V2X standard [13], [27]recommends a message frequency between 1 and 10 Hz. Totest the baseline performance, we let each vehicle broadcastVDUs at 10 Hz. From the POC test, we observe 10 objects perimage on average. Thus, the average VDU packet size is set to102 bytes. Table III and Table IV detail the CMR parametersand packet format. B. Results
Due to the dense traffic in the selected area, the averagevehicle speed in the simulation period is about km/h, https://data.gov.uk/dataset/gb-road-traffic-counts TABLE IV: Packet format ( B) Field Metadata Size (Byte)
Object ID position x position y velocity distance label confidence Vehicle IMUs Timestamp GPS
0° 90°180°270° -10 dBi0 dBi10 dBi
Fig. 6: Front-Rear antennae. Received BSMs CD F FrontRear_CMRFrontRear_Hop&DisFrontRear_Hop Generated BSMs CD F FrontRear_CMRFrontRear_Hop&DisFrontRear_Hop
Fig. 7: Generated and received BSMs with different filtermechanisms and antenna modes. Hop&Dis: filtered by hops(2) and distance (100m). Hop: filtered by hops (2).thus reflecting the lower bound of system performance underextremely congested scenarios.
Antenna type . As demonstrated by [48], [49], angled anten-nae, compared to idealistic isotropic antennas, can significantlychange the vehicular network dynamics. Therefore, we pro-pose a new antenna type, Front-Rear [44], to correspond withthe CMR protocol. Recall that CMR prioritizes data sent bysource vehicles traveling in a similar direction as the receiver.As shown in Figure 6, Front-Rear amplifies the signal in theforward and rear directions while reducing the transmissionrange on the sides. Hence, Front-Rear reduces the packets sentfrom vehicles driving on the sides and reduces the burden ofthe filters. Front-Rear can be deployed in a similar way asPatch [48], i.e., mounted to the front of the right and left sidemirrors and the right and left side of the rear windshield.
Filters . We compare CMR with other two classical filters (hopand distance limit (Hop&Dis), and hop limit only (Hop)). Fig-ure 7 shows the empirical cumulative distributions of the num-ber of generated and received basic safety messages (BSMs)by vehicles when using different filters. Figure 8 and Figure 9 .00 0.25 0.50 0.75 1.00
Packet loss ratio CD F FrontRear_CMRFrontRear_Hop&DisFrontRear_Hop
Fig. 8: Packet loss ratios with different filter mechanisms.Hop&Dis: filtered by hops (2) and distance (100m). Hop:filtered by hops (2).
Busy time CD F FrontRear_CMRFrontRear_Hop&DisFrontRear_Hop
Fig. 9: Busy time with different filter mechanisms. Hop&Dis:filtered by hops (2) and distance (100m). Hop: filtered by hops(2). TABLE V: Simulation results.
Filter Received BSMs Busy time (s) Packet loss
CMR 1024 0.39 0.63Hop&Dis 2630 1.01 0.54Hop 5652 2.41 0.71 show the empirical cumulative distributions of the packet lossratio and channel busy time experienced by the vehicles. Assummarized in Table V, CMR effectively filters considerablymore packets compared to hop&distance limit (-61%) or onlyhop limit (-81%). Furthermore, CMR has a slightly higherpacket loss ratio than hop&distance limit, but lower than hoplimit only. CMR also shows considerably less channel busytime than the other two filters (-61% and -83%, respectively).As mentioned in Section III, AICP focuses on informa-tiveness and thus employs CMR to filter low-informativenesspackets. We implement CMR in
Veins with only five lines ofcode and argue the CMR would also be lightweight in reality.Hence,
CMR could be easily integrated into routing protocolsfocusing on communication efficiency improvements .VIII. D
ISCUSSION
Due to the stochastic nature of human driving and thedriving environment, packets containing information about acertain object may not reach their destination consistently. Forinstance, vehicles may move in and out of the transmissionregion (hop limit), and thus only receive a fraction of thepackets concerning a given object. Similarly, the filtering al-gorithm may select different objects to display on each round.As such, the objects displayed on-screen may flicker and thus significantly degrade the driving experience by distracting thedriver and deteriorating the received information quality [50].As this contradicts the goals of our proposed AICP, infuture work we look to explore potential solutions such asan object persistence delay. In other words, once an object isdisplayed on-screen, the object remains displayed for a fixedamount of time, regardless of updates and potential filtering.This delay should be set to a value high enough in order notto distract the driver with high frequency flickering. However,longer delays may lead to a cluttering of the display withobjects of low informativeness. A delay between 500 ms to2 ms could represent an acceptable tradeoff to preserve highinformativeness while avoiding flickering.Beyond the object flickering, other user-centric and HCIaspects of the AICP system or potential system extensionscould also be a target of future work. In particular, wecould consider two different aspects. First, multi-modal cues(visual, audio, and tactile [35], [51]) could ease the driver’scognitive load and improve driving performance when thedriver’s attention primarily focuses on the road [52]. Second,we could evaluate the placement of certain visual contents(e.g., focal vs. peripheral placement [53], [54]) to determinethe optimal positioning for driving performance.Finally, our system relies on reasonable heuristics (e.g.,objects physically closer to the vehicle are more important) todetermine the importance of any specific object near the vehi-cle. However, given that a group of vehicles is an interactingset of agents, other methods might be helpful in predictingimportance in more complex situations, for instance, an acci-dent caused by a chain of actions that starts several cars away.Therefore, in future work we will examine a data-based deeplearning approach that accounts for such complex situations.The motivation for this approach also derives from researchshowing the benefit of deep learning in related advanced driverassistance systems (ADAS) [55].IX. C
ONCLUSION
In this work we propose AICP, the first solution that fo-cuses on optimizing informativeness for pervasive cooperativeperception systems with efficient filtering at both the net-work and application layers. To facilitate system networking,we also propose a networking protocol stack that includesVDU and CMR, a dedicated data structure and light-weightrouting protocol respectively, both specifically designed forinformativeness-focused applications. We also formulate theinformativeness problem in cooperative perception systemsfrom several different levels and propose a prioritized sortingalgorithm for fast information-based filtering. Overall, AICPdisplays only the most important information shared by nearbyvehicles and thus prevents information overload. We imple-ment a POC of the proposal with ARHUD and show thesystem has negligible additional processing latency ( . ms).Additionally, simulation results show that CMR effectivelyfilters less relevant packets, and thus considerably improvesthe channel availability of the vehicles. IEEE Intelligent Vehicles Symposium , 2012.[3] S.-W. Kim, B. Qin, Z. J. Chong, X. Shen, W. Liu, M. H. Ang, E. Fraz-zoli, and D. Rus, “Multivehicle cooperative driving using cooperativeperception: Design and experimental validation,”
IEEE Transactions onIntelligent Transportation Systems , 2014.[4] ETSI, “Intelligent transport systems (its); Cooperative Perception Ser-vices (CPS),” ETSI TS. 103 324, Early draft, 2020.[5] ——, “Intelligent transport system (its); Vehicular Communications;Basic Set of Applications; Analysis of the Collective-Perception Service(CPS),” ETSI TR. 103 562 v2.1.1, 2019.[6] H. Qiu, F. Ahmad, F. Bai, M. Gruteser, and R. Govindan, “Avr:Augmented vehicular reality,” in
MobiSys , 2018.[7] K. Garlichs, H.-J. G¨unther, and L. C. Wolf, “Generation rules for thecollective perception service,” in
VNC , 2019.[8] W. Liu, J. Gori, O. Rioul, M. Beaudouin-Lafon, and Y. Guiard, “Howrelevant is hick’s law for hci?” in
CHI , 2020.[9] R. Schweickert and B. Boruff, “Short-term memory capacity: magicnumber or magic spell?”
Journal of experimental psychology. Learning,memory, and cognition , 1986.[10] G. Thandavarayan, M. Sepulcre, and J. Gozalvez, “Analysis of messagegeneration rules for collective perception in connected and automateddriving,” in
IEEE Intelligent Vehicles Symposium , 2019.[11] S.-W. Kim, Z. J. Chong, B. Qin, X. Shen, Z. Cheng, W. Liu, and M. H.Ang, “Cooperative perception for autonomous vehicle control on theroad: Motivation and experimental results,” in
IROS , 2013.[12] H.-J. G¨unther, R. Riebl, L. Wolf, and C. Facchi, “Collective perceptionand decentralized congestion control in vehicular ad-hoc networks,” in
VNC , 2016.[13] ETSI, “Intelligent Transport Systems (ITS); ITS-G5 Access LayerSpecification for Intelligent Transport Systems Operating in the 5 GHzFrequency Band,” ETSI TR. 302 663 V1.3.0, 2019.[14] S.-W. Kim and W. Liu, “Cooperative autonomous driving: A mirror neu-ron inspired intention awareness and cooperative perception approach,”
IEEE Intelligent Transportation Systems Magazine , 2016.[15] S. Aoki, T. Higuchi, and O. Altintas, “Cooperative perception withdeep reinforcement learning for connected vehicles,” arXiv preprintarXiv:2004.10927 , 2020.[16] H. Qiu, F. Ahmad, R. Govindan, M. Gruteser, F. Bai, and G. Kar,“Augmented vehicular reality: Enabling extended vision for futurevehicles,” in
HotMobile , 2017.[17] Q. Chen, S. Tang, Q. Yang, and S. Fu, “Cooper: Cooperative perceptionfor connected autonomous vehicles based on 3d point clouds,” in
ICDCS’19 .[18] R. W. Wolcott and R. M. Eustice, “Fast lidar localization using mul-tiresolution gaussian mixture maps,” in
ICRA , 2015.[19] A. Y. Hata and D. F. Wolf, “Feature detection for vehicle localizationin urban environments using a multilayer lidar,”
IEEE Transactions onIntelligent Transportation Systems , 2015.[20] G. J. et al., “ultralytics/yolov5: v3.0,” Aug. 2020. [Online]. Available:https://doi.org/10.5281/zenodo.3983579[21] SAE International, “V2x communications message set dictionary,” 2020.[22] L. telcom, “In-car mobile signal attenuation measurements,”
LS telcomUK, Tech. Rep , 2017.[23] O. K. Tonguz, N. Wisitpongphan, and F. Bai, “Dv-cast: A distributedvehicular broadcast protocol for vehicular ad hoc networks,”
IEEEWireless Communications , 2010.[24] B. Karp and H.-T. Kung, “Gpsr: Greedy perimeter stateless routing forwireless networks,” in
MobiCom , 2000.[25] C. Lochert, M. Mauve, H. F¨ußler, and H. Hartenstein, “Geographicrouting in city scenarios,”
ACM SIGMOBILE mobile computing andcommunications review , 2005.[26] F. Granelli, G. Boato, D. Kliazovich, and G. Vernazza, “Enhancedgpsr routing in multi-hop vehicular communications through movementawareness,”
IEEE Communications Letters , 2007.[27] GSMA, “C-V2X; Enabling intelligent transport.” [28] I. Dirnbach, T. Kubjatko, E. Kolla, J. Ondruˇs, and ˇZ. ˇSari´c, “Methodol-ogy designed to evaluate accidents at intersection crossings with respectto forensic purposes and transport sustainability,”
Sustainability , 2020.[29] T. Higuchi, M. Giordani, A. Zanella, M. Zorzi, and O. Altintas, “Value-anticipating v2v communications for cooperative perception,” in . IEEE, 2019.[30] F. M. Favar`o, N. Nader, S. O. Eurich, M. Tripp, and N. Varadaraju, “Ex-amining accident reports involving autonomous vehicles in california,”
PLoS one , 2017.[31] R. De Maesschalck, D. Jouan-Rimbaud, and D. L. Massart, “The ma-halanobis distance,”
Chemometrics and intelligent laboratory systems ,2000.[32] H. Caesar, V. Bankiti, A. H. Lang, S. Vora, V. E. Liong, Q. Xu, A. Kr-ishnan, Y. Pan, G. Baldan, and O. Beijbom, “nuscenes: A multimodaldataset for autonomous driving,” in
CVPR , 2020.[33] B. Hannah Topliss, C. Harvey, and G. Burnett, “How long can a driverlook? exploring time thresholds to evaluate head-up display imagery,”in
AutomotiveUI , 2020.[34] F. Naujoks, Y. Forster, K. Wiedemann, and A. Neukum, “A human-machine interface for cooperative highly automated driving,” in
Ad-vances in Human Aspects of Transportation , 2017.[35] N. N. Correia and A. Tanaka, “Avui: Designing a toolkit for audiovisualinterfaces,” in
CHI , 2017.[36] G. Papadakis, D. Skoutas, E. Thanos, and T. Palpanas, “Blocking andfiltering techniques for entity resolution: A survey,”
ACM ComputingSurveys (CSUR) , 2020.[37] N. Cheng, F. Lyu, J. Chen, W. Xu, H. Zhou, S. Zhang, and X. S. Shen,“Big data driven vehicular networks,”
IEEE Network , 2018.[38] D. J. Sun, K. Zhang, and S. Shen, “Analyzing spatiotemporal traffic linesource emissions based on massive didi online car-hailing service data,”
Transportation Research Part D: Transport and Environment , 2018.[39] N. F. F. D. Silva, L. F. Coletta, and E. R. Hruschka, “A surveyand comparative study of tweet sentiment analysis via semi-supervisedlearning,”
ACM Computing Surveys (CSUR) , 2016.[40] I. Triguero, S. Garcia, and F. Herrera, “Self-labeled techniques for semi-supervised learning,”
Knowledge and Information systems , 2015.[41] B. Kulis et al. , “Metric learning: A survey,”
Foundations and Trends inMachine Learning , 2013.[42] E. Bochinski, V. Eiselein, and T. Sikora, “High-speed tracking-by-detection without using image information,” in
AVSS , 2017.[43] K. Q. Weinberger and L. K. Saul, “Distance metric learning for largemargin nearest neighbor classification.”
Journal of Machine LearningResearch , 2009.[44] P. Zhou, “AICP simulation code and datasets,” 2021. [Online].Available: https://github.com/pengyuan-zhou/AICP[45] C. Sommer, R. German, and F. Dressler, “Bidirectionally CoupledNetwork and Road Traffic Simulation for Improved IVC Analysis,”
TMC , 2011.[46] C. Sommer and F. Dressler, “Using the right two-ray model? a measure-ment based evaluation of phy models in vanets,” in
MobiCom , 2011.[47] C. Sommer, D. Eckhoff, and F. Dressler, “Ivc in cities: Signal attenuationby buildings and how parked cars can improve the situation,”
TMC’13 .[48] D. Kornek, M. Schack, E. Slottke, O. Klemp, I. Rolfes, and T. K¨urner,“Effects of antenna characteristics and placements on a vehicle-to-vehicle channel scenario,” in
ICC workshops , 2010.[49] D. Eckhoff, A. Brummer, and C. Sommer, “On the impact of antennapatterns on vanet simulation,” in
VNC , 2016.[50] A. Colley, J. H¨akkil¨a, B. Pfleging, and F. Alt, “A design space forexternal displays on cars,” in
AutomotiveUI , 2017.[51] S. Onimaru and M. Kitazaki, “Visual and tactile information to improvedrivers’ performance,” in
IEEE Virtual Reality Conference , 2010.[52] S. Onimaru, T. Uraoka, N. Matsuzaki, and M. Kitazaki, “Cross-modalinformation display to improve driving performance,” in
VRST , 2008.[53] R. H¨auslschmid, S. Osterwald, M. Lang, and A. Butz, “Augmenting thedriver’s view with peripheral information on a windshield display,” in
IUI , 2015.[54] I. Chaturvedi, F. H. Bijarbooneh, T. Braud, and P. Hui, “Peripheralvision: a new killer app for smart glasses,” in
IUI , 2019.[55] N. Formosa, M. Quddus, S. Ison, M. Abdel-Aty, and J. Yuan, “Predictingreal-time traffic conflicts using deep learning,”
Accident Analysis &Prevention , 2020.[56] W. Vazelhes, C. Carey, Y. Tang, N. Vauquier, and A. Bellet, “metric-learn: Metric learning algorithms in python,”