Object Removal Attacks on LiDAR-based 3D Object Detectors
Zhongyuan Hau, Kenneth T. Co, Soteris Demetriou, Emil C. Lupu
OObject Removal Attacks on LiDAR-based 3D ObjectDetectors
Zhongyuan Hau § , Kenneth T. Co § , Soteris Demetriou, Emil C. Lupu Imperial College London { zy.hau17, k.co, s.demetriou, e.c.lupu } @imperial.ac.uk Abstract —LiDARs play a critical role in Autonomous Vehicles’(AVs) perception and their safe operations. Recent works havedemonstrated that it is possible to spoof LiDAR return signalsto elicit fake objects. In this work we demonstrate how thesame physical capabilities can be used to mount a new, evenmore dangerous class of attacks, namely
Object Removal Attacks (ORAs). ORAs aim to force 3D object detectors to fail. Weleverage the default setting of LiDARs that record a single returnsignal per direction to perturb point clouds in the region ofinterest (RoI) of 3D objects. By injecting illegitimate points behindthe target object, we effectively shift points away from the targetobjects’ RoIs. Our initial results using a simple random pointselection strategy show that the attack is effective in degradingthe performance of commonly used 3D object detection models.
I. I
NTRODUCTION
We are currently undergoing a revolution in transportationand mobility. New generations of vehicles are increasinglyequipped with high-precision depth sensors to better perceivetheir environment and offer unprecedented levels of driverassistance and driving autonomy. Such vehicles commonlyrely on LiDAR sensors, which collect high definition depthmeasurements stored in 3D point clouds. Reliably detectingobjects from such point clouds is vital to the safety of theautonomous vehicle, its users and passengers.
LiDAR spoofing attacks.
Recent studies have shown thatit is possible to attack LiDAR-based perception systems byspoofing LiDAR return signals [6], [1], [10], [5]. Petit et al. first demonstrated this with physical attacks that can injectup to 10 fake 3D points in a point cloud. Cao et al. andSun et al. progressively improved on the physical capabilitiesof the LiDAR spoofing adversary showing that one couldreliably inject up to 60 and 200 fake points respectively. Moreimportantly, Cao et al. developed a white-box model-leveldigitally simulated LiDAR spoofing attack that can introducefront-near fake measurements in a scene, which are thendetected as objects by an end-to-end autonomous vehicle (AV)system. Sun et al. then demonstrated both white-box andblack-box attacks that spoof vehicles in front-near locations byexploiting patterns of occluded and distant vehicles. Xiang etal. [13] examined the vulnerability of point-cloud based object § Equal contribution detectors and proposed white-box approaches for point shiftingand point injection to craft adversarial point-clouds. Zhao et al. [14] proposed a class of point cloud perturbation attacks thatminimizes the number of points perturbed to flip the resultsof point-cloud based object detectors. They use gradient-basedand genetic algorithm approaches to generate adversarial pointclouds with perturbations of up to 150 points to subvert objectdetection with a 95% success rate.
Object hiding. Tu et al. [12] proposed both white-box andblack-box methods to generate adversarial objects that whenplaced above a target vehicle, would evade point-cloud basedobject detectors with a success rate of 80%. For the white-boxattack, the adversarial object is generated using a gradient-based approach to minimize the confidence score of the targetobject (vehicle). A black-box attack was also demonstrated,where the adversarial objects are chosen using a geneticalgorithm approach to iterate and improve adversarial objectmeshes. Object-hiding attacks are considered more dangerousthan spoofing objects. Whilst detecting a spoofed object canbring the ego-vehicle to a full stop, failing to detect an objecthas a higher chance of leading to a fatal collision. Our work . We leverage the demonstrated state of the art ca-pabilities of the physical LiDAR spoofing adversary [6], [10],[1] to design a new model-level object removal attack (ORA)that aims to hide objects from 3D object detectors. Comparedto prior work with spoofed objects [6], [10], [1] ORAs have adifferent goal: the adversary aims not to introduce a ghost ob-ject but force mis-detection of a genuine object which can havemore severe consequences. Moreover, in contrast with relatedwork on 3D object hiding [12], we introduce a new techniquethat does not aim to introduce patterns on top of genuineobjects, but rather to spoof points within a genuine object’sbounding box such that they appear away from their originalposition and cause object mis-detection. ORAs are stealthiersince they do not require placing adversarial objects on thetarget, are easier to mount and have high success rates. Weconduct digital ORA attacks, emulating the physics of LiDARoperation. Their effectiveness is evaluated against popular 3Dobject detectors (PointRCNN [8] and Point-GNN [9]).We found that an adversary with the ability to inject ≤ points can reduce their recall to less than 25% for Pedestrian and
Cyclist object classes on both models with a random point selection strategy. Our work demonstrates thefeasibility of ORAs and we hope to inspire future work onmore sophisticated ORA strategies that can lead to a betterunderstanding of the LiDAR spoofing adversary model. a r X i v : . [ c s . C V ] F e b ABLE I. A
VERAGE P RECISION (AP) OF OBJECT DETECTION FOR DIFFERENT CLASSES UNDER THE
ORA
RANDOM ATTACK .Model Attack Budget Car AP (IoU = 0.7) Pedestrian AP (IoU = 0.5) Cyclist AP (IoU = 0.5)Easy Moderate Hard Easy Moderate Hard Easy Moderate Hard0 (Clean) 88.86 78.61 77.75 62.87 54.88 48.95 73.08 56.33 52.3610 79.30 65.09 58.11 50.69 42.96 38.15 61.03 39.04 35.5820 78.85 59.57 51.00 48.06 40.38 35.42 54.66 31.67 30.12PointRCNN [8] 40 77.02 54.36 46.13 43.66 35.84 31.96 41.81 24.36 22.9660 72.97 48.01 40.42 38.70 31.97 27.82 33.54 19.14 18.64100 64.69 40.97 33.47 35.99 28.46 23.20 24.48 15.75 15.77150 55.74 34.05 28.33 29.04 22.79 19.93 17.01 11.95 11.33200 47.32 31.05 25.00 25.62 19.02 16.92 13.07 9.43 9.420 (Clean) 89.89 88.82 87.75 73.74 70.34 63.57 85.69 64.44 62.2510 89.62 78.33 69.31 73.74 70.25 63.49 85.69 64.17 62.0120 89.34 71.38 62.06 71.89 64.01 61.19 76.79 52.78 48.32Point-GNN [9] 40 86.40 62.50 53.08 63.18 55.57 48.28 57.00 36.70 34.8560 80.26 53.71 44.60 56.27 48.10 44.42 43.45 26.98 26.04100 70.47 43.89 35.36 47.15 39.24 32.50 25.63 16.06 15.86150 57.15 34.61 29.17 39.23 31.65 27.72 10.86 8.14 7.40200 45.58 28.01 22.01 31.99 26.64 23.81 4.76 3.66 3.85
II. O
BJECT R EMOVAL A TTACK
We introduce a new class of attacks, namely object removalattacks (ORAs). ORAs are black-box, model-level attacks thatcan be launched by a physical LiDAR spoofing adversary. Thegoal of ORAs is to displace the original depth measurementswithin a genuine object’s bounding box to appear outside thatbounding box such that the target object is not detected by 3Dobject detectors. Below we elaborate on our threat model anda preliminary strategy to demonstrate the feasibility of ORAs.
A. Threat Model
Physical Capabilities.
We assume an adversary A , who canspoof the LiDAR return signals of a target AV [1], [6], [10],[11]. The adversary can achieve this by deploying a devicewithin the line of sight of a victim vehicle’s LiDAR sensor.The adversarial device can capture LiDAR signals, alter themand emit them toward the victim sensor with a controlleddelay. By controlling the return signal, A can manipulate theresulting 3D measurements reported in a 3D point cloud by thevictim sensor. We assume A has state of the art sensor spoofingcapabilities and can inject ≤ points in a 3D scene withina horizontal angle of 10 ◦ [11]. We also assume that A canspoof resulting measurements to appear further away from thetarget vehicle than they actually are [10]. Digital Capabilities.
Since A is in physical proximity, weassume it can also sense the environment and detect objectswithin the vicinity of the target vehicle. Using basic transfor-mations, A can change the coordinate system of a 3D scenefrom the reference point of A to that of the target vehicle. B. Object Removal Attack (ORA) Rationale
LiDAR spoofing has been previously demonstrated [1],[10], [11]. These attacks exploit the fact that LiDARs deployedin AVs operate under the Strongest Return Mode setting,where only a single measurement can be recorded per raydirection. Therefore, the injection of a spoofed point resultsin the original corresponding point in the direction of the ray(from the LiDAR to the spoofed point) to be displaced. Wealso leverage this phenomenon, but to achieve a different goal.Instead of aiming to introducing objects, we highlight a newclass of attacks, namely
Object Removal Attacks (ORAs) thataim to remove objects. Our proof of concept hides an object
Algorithm 1
Obtaining Attack Trace (ORA-Random)
Input: list( obj pts coords ) (cid:46) Target Object’s Point Cloud candidate pts coords =[] attack trace pts coords = [] for each pt in obj pts coords doif pt within spoofing_horizontal_angle then candidate pts coords ← pt end ifend for attack pts ← random( candidate pts coords , A budget ) attack trace pts coords ← ( pts in obj pts coords ∧ notin attack pts ) for each pt in attack pt do attack pt coords ← dist increment along ray( pt ) attack trace pts coords .append( attack pt coords ) end forreturn attack trace pts coords from AV perception by displacing points from a target object’spoint cloud with LiDAR point injection behind that object. C. ORA Operating Mechanism
An ORA exploits the LiDAR’s default mode for recordingthe measurements of return signals–where a single returnsignal per ray direction is recorded. This enables an adversaryto perform point injections that can remove points from atarget object’s original point-cloud by spoofing another signalat a different location in the same direction of the raysthat are incident on an object. The resulting perturbed point-cloud would cause point-cloud based object detectors to missdetecting the target object, and thus evading object detection.The process of ORA first starts with the adversary iden-tifying the target object’s location and the region of interest(RoI) where points are to be selected from and removed. Here,we assume that the adversary has knowledge of the 3D sceneand is able to obtain bounding boxes of target objects and thecoordinates of the points in the bounding boxes (i.e. objectpoints). This can be achieved by finding a translation matrixthat changes the coordinates system from the reference point ofthe attacker to the ego-vehicle [3]. Next the adversary obtains aset of points from the object points that are within its spoofinghorizontal angle (i.e. candidate points); one way of achieving2 ig. 1. Recall-IoU curves for (top) PointRCNN and (bottom) Point-GNN under the ORA random attack with different IoU thresholds. this is to segment the object bounding box by the spoofingangle and only use the points within the segment. In ourcase, we used the left-most coordinates of the bounding boxas an anchor point to calculate the points that are within thehorizontal spoofing angle. Lastly from the subset of candidatepoints within the spoofing angle, the adversary picks pointswithin its budget to perform point injection, spoofing points ata random distance behind the original points’ location (in thedirection of the ray). ORA is modular and can be used withvarious point selection strategies. We demonstrate ORA witha random point selection (
ORA-Random ) from the candidatepoints.
ORA-Random is detailed in Algo. 1.III. E
XPERIMENTS & R
ESULTS
Models & Datasets.
The proposed attack was conducted onthe validation set (3769 out of 7481 scenes) of the KITTIdataset [4]. Objects in these scenes are subjected to
ORA-Random and the resulting perturbed point clouds of the scenesare subsequently passed to popular 3D object detectors toevaluate the performance of the attack. We perform the at-tacks on three object types,
Cars , Pedestrians and
Cyclists asthese objects are commonly encountered in AV scenarios. Weperform our attack evaluations on widely-used models for 3Dpoint cloud object detection that rely solely on LiDAR data,Point-GNN [9] and PointRCNN [8]. The two models differ inhow feature extraction is performed for the object detectiontask. Point-GNN uses a graph neural network that encodesthe point-cloud directly as a graph representation for objectdetection. Whereas, PointRCNN uses a multi-layer perceptronlearning approach on point sets to obtain point feature vectorsof the point cloud(PointNet++ [7]), which are then furtherprocessed for object detection.
Performance Metrics.
For all models and scenarios, wemeasure the 3D AP and Recall-IOU curves of the modelsunder attack. The 3D AP (average precision) captures theratio of true positive predictions over all positive predictionsand is the primary measure for overall performance of 3Dobject detectors. The Recall-IOU curve measures the recallof the detector for various IOU thresholds. The goal of theattacker would be to hide real objects from the model, so themeasurement of recall is relevant in our case since it captures the ability of the detector to not miss objects. Therefore, A ’sgoal would be to lower recall scores for target detectors. Evaluation Scenarios.
We consider two scenarios for evalu-ating the models’ performance when under attack. The firstis on the performance of the attack applied on the entireKITTI validation dataset. The second evaluation focuses onthe impact of ORAs on the detection of front-near objects.For both scenarios, we use
ORA-Random to perturb the pointcloud of each individual type of target object found in thescenes with various point-perturbation budgets (10, 20, 40, 60,100, 150, 200 points) that are within A ’s capabilities. A. Attack Performance Evaluation
Table I shows the AP of the 2 models for clean (no pointperturbation) and for the various attack budgets used to perturbthe 3 object types. The evaluation criteria follows that of theKITTI 3D object detection benchmark (where the detectiondifficulty levels are determined by the size and occlusion ofobject). We observed that the AP decreases for increasing A budget . The effect of ORA-Random is most significant for
Cyclist objects and then followed by
Pedestrian and then
Car objects. One reason could be that Cyclists are not ascommon in the dataset compared to the other two classes,resulting in poorer performance.
Pedestrian and
Cyclist objectsare smaller objects and have significantly fewer points in theirpoint clouds.From Fig. 1, we observed that when increasing the pointbudget, the recall falls. For Cars at IoU ≥ Pedestrians and
Cyclists at IoU ≥ ORA-Random attack is very effective in degrading the objectdetector’s performance and hiding a target object.B. Attacking Front-Near Objects
We further investigate whether ORAs can mask front-near objects (objects in close proximity to the ego-vehicle),where accurate detection is critical to the safe operation of theautonomous vehicle. Our results are summarized in Fig. 2. Weobserve a general trend where objects further away from the3 ig. 2. Recall metrics by distance (in metres away from the LiDAR) for different IoU configurations of (top) PointRCNN and (bottom) Point-GNN.
LiDAR have lower recall but with the effects being visible evenfor objects ≤ m . The extent of the drop in recall (w.r.t clean)is also correlated to the increase in A ’s budget. Noticeably forsmaller objects such as Pedestrian and
Cyclist , we observe ahigher decrease in recall when increasing A budget and distance.This is due to the smaller objects’ inherent low number ofpoints and its decreasing point density as distance increases.This provides an opportunity for the adversary to use its limited A budget to perturb a larger proportion of points in the object’spoint cloud–increasing its success rate of evading detection. C. Discussion & Implications
Although
ORA-Random does not drastically damage therecall for front-near objects, it is still able to significantly lowerrecall and AP when considering the more general validationdataset. The detection of further away objects remains acritical function especially during the high-speed operationof vehicles. Thus being able to damage model recall in thegeneral case with a random point selection strategy raises gravesecurity concerns. Additionally, the random point selection canbe improved upon with more optimized strategies that usemethods such as genetic, evolutionary, or Bayesian algorithms[2], [12], [14] to create effective adversarial attacks within apoint budget. Overall
ORA-Random demonstrates that objectremoval attacks are a real concern which we plan to investigatein depth in future work.IV. C
ONCLUSION
In this paper, we provide preliminary evidence that with asimple approach of shifting 3D points from a RoI, a LiDARspoofing adversary is able to effectively perturb the point cloudof a target object to render it undetectable. We performeda sensitivity analysis and found that for smaller objects, theattacks are highly effective at a distance beyond 11m. Thisposes a safety concern as failure to detect such objects couldhave life-threatening consequences. In future work, we plan toimplement optimization-based point selection ORA strategies,verify the feasibility of ORAs in the physical domain and studythe effect of ORA at various distances and driving speeds onAV driving decisions with an AV simulator. As this new classof attack targets a single sensor modality, we are exploringdefenses using multi-sensor fusion with RGB cameras. R
EFERENCES[1] Yulong Cao, Chaowei Xiao, Benjamin Cyr, Yimeng Zhou, Won Park,Sara Rampazzi, Qi Alfred Chen, Kevin Fu, and Z Morley Mao.Adversarial sensor attack on lidar-based perception in autonomousdriving. In
Proceedings of the 2019 ACM SIGSAC Conference onComputer and Communications Security , pages 2267–2281, 2019.[2] Kenneth T. Co, Luis Mu˜noz Gonz´alez, Sixte de Maupeou, and Emil C.Lupu. Procedural noise adversarial examples for black-box attacks ondeep convolutional networks. In
ACM Conference on Computer andCommunications Security , CCS ’19, pages 275–289, 2019.[3] Sundaram Ganapathy. Decomposition of transformation matrices forrobot vision.
Pattern Recognition Letters , 2(6):401–412, 1984.[4] Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun.Vision meets robotics: The kitti dataset.
International Journal ofRobotics Research (IJRR) , 2013.[5] Zhongyuan Hau, Soteris Demetriou, Luis Mu˜noz-Gonz´alez, and Emil C.Lupu. Shadow-catcher: Looking into shadows to detect ghost objectsin autonomous vehicle 3d sensing. arXiv preprint arXiv:2008.12008 ,2021.[6] Jonathan Petit, Bas Stottelaar, Michael Feiri, and Frank Kargl. Remoteattacks on automated vehicles sensors: Experiments on camera and lidar.
Black Hat Europe , 11:2015, 2015.[7] Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas.Pointnet++: Deep hierarchical feature learning on point sets in a metricspace. In
Advances in Neural Information Processing Systems , pages5099–5108, 2017.[8] Shaoshuai Shi, Xiaogang Wang, and Hongsheng Li. Pointrcnn: 3dobject proposal generation and detection from point cloud. In
IEEEConference on Computer Vision and Pattern Recognition , pages 770–779, 2019.[9] Weijing Shi and Raj Rajkumar. Point-gnn: Graph neural network for3d object detection in a point cloud. In
IEEE Conference on ComputerVision and Pattern Recognition , pages 1711–1719, 2020.[10] Hocheol Shin, Dohyun Kim, Yujin Kwon, and Yongdae Kim. Illusionand dazzle: Adversarial optical channel exploits against lidars forautomotive applications. In
International Conference on CryptographicHardware and Embedded Systems , pages 445–467. Springer, 2017.[11] Jiachen Sun, Yulong Cao, Qi Alfred Chen, and Z. Morley Mao. Towardsrobust lidar-based perception in autonomous driving: General black-box adversarial sensor attack and countermeasures. In , pages 877–894. USENIXAssociation, August 2020.[12] James Tu, Mengye Ren, Siva Manivasagam, Ming Liang, Bin Yang,Richard Du, Frank Cheng, and Raquel Urtasun. Physically realiz-able adversarial examples for lidar object detection. arXiv preprintarXiv:2004.00543 , 2020.[13] Chong Xiang, Charles R Qi, and Bo Li. Generating 3d adversarialpoint clouds. In
IEEE Conference on Computer Vision and PatternRecognition , pages 9136–9144, 2019.[14] Yiren Zhao, Ilia Shumailov, Robert Mullins, and Ross Anderson. Nudgeattacks on point-cloud dnns. arXiv preprint arXiv:2011.11637 , 2020., 2020.