Automated Fuzzing of Automotive Control Units
Timothy Werquin, Mathijs Hubrechtsen, Ashok Thangarajan, Frank Piessens, Jan Tobias Muehlberg
AAutomated Fuzzing of Automotive Control Units
Timothy Werquin, Mathijs Hubrechtsen, Ashok Thangarajan, Frank Piessens and Jan Tobias M¨uhlberg
KU Leuven, Dept. of Computer Science, imec-DistriNet
B-3001 Leuven, Belgium (cid:104) firstname.lastname (cid:105) @cs.kuleuven.be
Abstract —Modern vehicles are governed by a network ofElectronic Control Units (ECUs), which are programmed to senseinputs from the driver and the environment, to process theseinputs, and to control actuators that, e.g., regulate the engine oreven control the steering system. ECUs within a vehicle commu-nicate via automotive bus systems such as the Controller AreaNetwork (CAN), and beyond the vehicles boundaries throughupcoming vehicle-to-vehicle and vehicle-to-infrastructure chan-nels. Approaches to manipulate the communication betweenECUs for the purpose of security testing and reverse-engineeringof vehicular functions have been presented in the past, all ofwhich struggle with automating the detection of system changein response to message injection. In this paper we presentour findings with fuzzing CAN networks, in particular whileobserving individual ECUs with a sensor harness. The harnessdetects physical responses, which we then use in a oracle functionsto inform the fuzzing process. We systematically define fuzzers,fuzzing configurations and oracle functions for testing ECUs.We evaluate our approach based on case studies of commercialinstrument clusters and with an experimental framework forCAN authentication. Our results show that the approach iscapable of identifying interesting ECU states with a high levelof automation. Our approach is applicable in distributed cyber-physical systems beyond automotive computing.
Index Terms —automotive control networks, CAN, security,testing, fuzzing
I. I
NTRODUCTION
Modern cars are largely controlled by software. This softwareforms a distributed mixed-criticality system that executeson a number of interconnected Electronic Control Units(ECUs). Jointly, these ECUs govern the vehicle’s behaviour– from convenience and infotainment functions to safety-critical functionality. ECUs are connected via automotive bussystems that facilitate the exchange of messages, most of whichcommunicate sensor readings and control instructions. Thecontrol software then interprets these messages and reacts toevents by triggering the relevant actuators, e.g., brakes, airbags,or steering gear. In 2016, the Ford Motor Company reportedthat their latest models are running on 150 million lines ofcode. Given the enormous complexity of these systems, theyare notoriously hard to test, for safety as well as for securityproperties.Since around 2004, researchers have been expressing theirconcerns with respect to the security limitations of com-munication standards, including the widely used ControllerArea Network (CAN) [1]–[3]. Since 2010, a series of high-profile attacks [4]–[7] illustrate that with increased vehicularconnectivity even remote adversaries can take control of criticalfunctions of a vehicle. These risks have been acknowledged and are partly addressed in emerging industry standards [8],[9] that encompass authentication and software security forcontrol systems, and prototypes that showcase secure systemdesigns for automotive computing based on software attestationand Trusted Computing primitives [10] have been proposed.Meanwhile, more and more low-level vulnerabilities in thesecommunication systems are being revealed (e.g., [11] and [12]),guidance for the reverse-engineering and penetration testing ofvehicular communications and control systems becomes readilyavailable [13], and the need for advanced testing methodologyfor these systems is generally acknowledged. A testing approachthat promises a particularly high level of automation is fuzzing.Fuzz testing [14], [15] is a well established methodologyto expose software and systems to unexpected conditions, forexample by providing random input streams that may crash thetarget. Yet, the approach does not easily apply to embeddedsoftware [16] and few approaches have been made to fuzzembedded control systems or automotive ECUs in particular [5],[17], [18]. A key difficulty to overcome here is the definitionof oracle functions that define when a fuzzer has potentiallytriggered a bug or at least an “interesting” system state, andto automatically evaluate these functions.
1) Our Contributions.:
In this paper we discuss fuzz testingin the context of automotive control networks. Specifically, ourresearch investigates the use and automation of fuzzing so as tofind vulnerabilities and to reverse engineer ECU functionalityin CAN networks. We make the following contributions:1) We systematically define fuzzers, fuzzing configurationsand oracle functions for testing automotive ECUs throughtheir CAN interface.2) We develop a sensor harness to automatically evaluatefuzzing oracles for ECUs with physical outputs.3) We evaluate our approach, taking commercial automotiveinstrument clusters and an experimental setup for testingAUTOSAR-compliant message authentication as casestudies.To the best of our knowledge, this paper is the firstto largely automate a methodical fuzzing approach (e.g.following [15]) for automotive ECUs. Although our imple-mentation is targeting CAN components, our approach canbe generalised to cyber-physical systems with any underly-ing communication technology. Our fuzzer implementation,instructions to build the sensor harness and to repeat ourexperiments are available under an open-source license athttps://github.com/timower/caringcaribou/tree/autoFuzz. a r X i v : . [ c s . CR ] F e b I. B
ACKGROUND
In this section we briefly introduce the CAN bus, which iscommonly used to facilitate communication between automo-tive ECUs but also in industrial control systems. We furtherintroduce the CaringCaribou penetration testing framework,which our fuzzing toolchain is integrated with.
A. Controller Area Network (CAN) & Security
The CAN bus is the most commonly used broadcast networkin modern cars. A CAN message consists of an 11-bitarbitration ID, followed by an optional 18-bit extended ID, andup to 8 bytes of data payload (cf. Fig. 1). Dedicated transceiverhardware implements a protocol for message acknowledgementand bus arbitration for sending/receiving data frames. CANrequires a fixed data transmission rate, and allows recessivebits (one) to be overwritten by dominant bits (zero) duringtransmission. Message acknowledgement can thus simply beimplemented by overwriting the ACK bit at the end of thedata frame in real-time. Likewise, to implement bus arbitration,CAN transceivers are required to continuously listen on thebus while sending the message ID at the beginning of the dataframe, and to back off when one of their ID bits has beenoverwritten. This scheme ensures that messages with lowerIDs effectively have higher priorities. Finally, each CAN framefeatures a 16-bit CRC field to detect transmission errors.
Figure 1. Extended data frame standardised by CAN 2.0B.
CAN was originally developed in 1983, when cyber attackswere of no concern. Thus, the protocol does not provide anyform of message authentication. Any ECU connected to thenetwork can spoof messages with arbitrary sender ID andpayload, which forms the basis of many attacks [4]–[7]. As aresponse, the AUTOSAR [9] standardisation body publishedindustry guidelines for backwards-compatible message authen-tication in vehicular networks. The VulCAN framework [10],which we study in Sect. V-A is one implementation of theseauthentication extensions.
B. CaringCaribou and Automotive Penetration Testing
We built our implementations of CAN fuzzers as modulesfor the open-source penetration-testing framework CaringCari-bou. Our fuzzing extensions are freely available for furtherexperimentation and follow-up research. Since fuzzers arewidely used as a means to perform black-box testing in theregular penetration testing industry, it is our belief that a similartool could prove useful in the automotive penetration testingcommunity.CaringCaribou is a tool developed for the purpose of beingthe “nmap” of automotive security. Cyber security research inthe automotive industry is a new field that is rapidly expanding.Yet, it still lacks the mature tooling available to the mainstream https://github.com/CaringCaribou/caringcaribou security community. Tools like CaringCaribou aim to fill thatgap. CaringCaribou has a modular architecture that allowsdevelopers of penetration testing techniques for automotivesystems to easily write new modules for their specific purpose,and deploy these modules using a unified tooling infrastructure.Thus, CaringCaribou provides the developer with a layer ofabstraction which protects them from the specifics of CANand other automotive communication protocols, allowing themto focus on writing the actual penetration testing tool insteadof dealing with the lower-layer interactions. CaringCaribou issupposed to be a zero-knowledge tool that can be deployed onany CAN network regardless of its specific configuration.III. F UZZING
CAN N
ETWORKS
In this section we lay out our approach to define a fuzzingtool for CAN-based automotive control systems. We follow theapproach of Manes et al. [15] and dissect the tool into an oraclecomponent, the actual fuzzer and the run-time configurationfor the fuzzer for a particular run.
A. Bug Oracles for ECUs
According to [15], a (bug) oracle is a program that determineswhether a given execution of the target system violates aspecific (security) policy. In Sect. V we outline two verydifferent case studies for our system: In one of these we havea partial specification of security properties of the networkavailable, and where we are looking for violations of thisspecification. In the second case study we have no reliablespecification but we are interested in reverse-engineering sucha specification. Both case studies are characterised by not beingable to observe software interactions directly (as opposed tosoftware fuzzing with code instrumentation in a simulator [15]).Instead, we are looking at black-box systems to which ourfuzzer can provide an input stream, while any observablecommunication output, physical output (actuation of a LED ora relay) or even the timing or absence of such outputs (e.g.,due to a software crash) may indicate that an “interesting”system state has been reached.Recent related work in the field of intrusion detection forindustrial control system (e.g., [19] and [20]) suggests thatmachine-learning approaches can be used to train detectorsthat then report anomalies in the communication behaviourof vehicular networks. We have not implemented such oraclefunctions.Physical outputs of control units can certainly be observed byhuman operators. They can also be sensed and electronicallyreported through sensor networks, or in our case a sensorharness that is attached to the target system. In the followingsections we emphasise on this form fuzzing oracle, where astate change in the target is defined by a sensor (de-)activationor sensor threshold.Most difficult is certainly the detection of system failurewhich results in the absence of an observable response from thetarget. Thus, inputs that lead to failures are easily misinterpretedby a fuzzing tool as inputs that have no effect. For example,our work deals with ECUs that need to regularly receiveertain messages or they will fault, effectively rendering thecontinuation of a fuzzing campaign ineffective. Detectingthese messages and constructing traffic that satisfies the inputrequirements of ECUs in the absence of a specification, isdifficult. An oracle to identify these kinds of system faultsrequires the use of recorded traffic that periodically triggers aknown observable output. An oracle function will then fire whenthe periodic signal is absent due to, e.g., message omission.We apply this method in our omission fuzzer below.
B. Defining CAN Fuzzers
Fuzzing is the execution of the target system using input(s)sampled from an input space (the “fuzz input space”) thatprotrudes the expected input space of the target system [15].With fuzzing we aim to enumerate and exercise a large subsetof this fuzz input space to find system behaviour that triggersan oracle function. ECUs that process CAN messages arean interesting target since the frame size of CAN messagesis at most 110 bits. This fuzz input space is certainly huge,but much smaller than, e.g., Ethernet frames, WiFi frames,or multimedia streams. Still, even for CAN networks, thisfuzz input space is prohibitively large for being exhaustivelyexercised. Furthermore, with a maximum bandwidth of 1 MBit,and most ECUs using 500 MBit as a fixed transfer rate,data transmission to a target network of ECUs represents abottleneck.Starting with the idea of random fuzzing , where arbitra-tion IDs and message payloads are selected randomly, wedevise three additional fuzzing strategies, brute-force fuzzing , mutation fuzzing and identify fuzzing , to narrow down thefuzz input space and explore interesting ECU behaviour moreefficiently. These strategies are based on the observationthat an ECU typically accepts inputs on a relatively smallnumber of IDs only, that also the number of payload bitsthat result in a observable state change is limited, and thatseveral consecutive messages may be required to trigger anobservable state change. We then integrate these approaches inan automated exploration mode, where inputs from a sensorharness (cf. Sect. IV), which is attached to a target ECU,guide input generation. We have implemented our approachin two modules for CaringCaribou, namely fuzzer and autoFuzz , which can be invoked as ./cc.py
1) Brute-Force Fuzzing.:
This method aims to exhaustivelyenumerate selected hexadecimal digits in a message, specificallyin the message’s ID field and the payload. For example,the fuzzer can be invoked as ./cc.py fuzzer brute0x123 12ab..78 , where the 5th and 6th octet of themessage payload will be enumerated and sent, while the message ID and all other payload octets remainconstant.
2) Mutation Fuzzing.:
This strategy can be used to systemati-cally explore a larger fuzz input space through mutating selectedhex digits in arbitration ID and message by means of individualrandom bit flips. An example use for this strategy is ./cc.pyfuzzer mutate 7f.. 12ab.... ; the syntax followsthe example given for brute-force fuzzing above.
3) Identify Fuzzing.:
Once a fuzzing run resulted in an eventof interest, e.g., a change of an indicator LED on a target ECU,the identify method can be used to replay and identify a minimalset of messages that caused the event. The syntax for invokingthis method is ./cc.py fuzzer identify log.txt ,where log.txt refers to a log file previously recorded withthe -f parameter. The method relies on human input – i.e., keypresses – to gather information about the timely occurrenceof events, and aims to prune the set of recorded messages in log.txt so that the event still occurs when the pruned setis replayed.
4) Automated ECU Exploration.:
Our autoFuzz moduleimplements the above strategies so that system change canbe detected directly through our sensor harness (cf. Sect. IV).Sensor observations can then be used to guide the generation ofthe next inputs and to automatically identify message bits thatlead to observable system change, depending on the fuzzingstrategy. The module further features the generation of J1939-compliant messages and the fuzzing of J1939 function groupaddresses (PGNs and SPNs).When fuzzing an ECU with a single sensor attached toone of the ECU’s actuators, it is possible to immediatelyrun the identify fuzzer when a change in the sensor stateis detected without relying on recorded traffic. This requiresthat the actuator can be triggered with a predictable payload.When using multiple sensors, the log file can be filtered to keepa number of messages preceding the activation of a specificsensor which can then be used as input to the identify fuzzer.While experimenting with fuzzing strategies, we observedthat an ECU’s response to a message is often delayed. Duringthe delay period, other messages are being sent by thefuzzer, which makes identifying the CAN messages specificallyresponsible for an response more difficult. One possible solutionis to increase the delay between sending messages. Yet, thiswill increase the time required to cover the fuzz input space.Another option is the resend only a subset of the messagespreceding a sensor activation with increased delays, which weimplemented in our identify method.Our experiments further revealed that some ECUs expectcertain messages to be received regularly. The absence ofthese messages will lead to a shut-down or render the ECUunresponsive and to indicate a failure. These behaviours preventour identify method from working as sending the completetraffic log can keep the ECU responsive but sending parts ofthe recorded traffic will cause the ECU to fault. To addressthis, we developed an approach that we refer to as omissionfuzzing . This strategy sends the complete recorded traffic butomits some messages in order to identify which message causepecific state changes. The identified arbitration IDs or payloadscan then be added to a “blacklist” to inform other methods.E.g., any arbitration ID in the blacklist will never be omittedduring identify fuzzing .After collecting logs from the fuzzer, various analysis canbe applied. For example, if a specification of the target ECU isavailable, which would detail the expected actuator responsesto specific groups of messages, sensor activations can becheck against this specification to detect bugs or undocumentedbehaviour.
C. Target-Specific Fuzzer Configuration
A fuzz configuration of a fuzzer comprises the parametervalue(s) that control(s) the fuzz algorithm [15]. In the contextof our approach, these parameters involve message generation , message timing , message omission , and the configuration ofthe sensor harness . As outlined before fuzzing entire CANmessages makes little sense as it results in an extremely largefuzz input space. Thus, configurations will typically restrictthe fuzz space to specific octets in (extended) arbitration IDsand message payload. Message timing is typically configuredto schedule a new message every 3 ms to 20 ms to avoidmessage collisions and to leave enough time for actuators tobe engaged and sensed. Message omission and baseline trafficare to be set up to simulate typical bus traffic in a car so asto make target ECUs function normally. The sensor harnessoffers a wide range of configuration options that involve thetype of sensors, sampling rates, the number of sensors andtheir placement on the target ECU.IV. A S ENSOR H ARNESS TO A UTOMATE
ECU F
UZZING
In this section we describe an inexpensive and extensibleexperimental sensor harness to automate the analysis ofautomotive ECUs. The intuition behind the setup is that fuzzingcommunication in an automotive control network, or in cyber-physical systems in general, can cause a range of interestingresponses beyond network communication. Thus, to use theseresponses as inputs to fuzzing oracles (cf. Sect. III-A) theymust be automatically measured at an appropriate samplingrate. Previous approaches to consider these responses typicallyrely on human observation and human interaction during thefuzzing process. For example, a fuzzing tool may require theuser to press a key if they observe a change in the system,e.g., a flashing indicator light on a control panel. Our workimproves over this by detecting physical responses of ECUsautomatically, with negligible delays, and at a configurablegranularity.Fig. 2 gives an overview of the sensor harness. The systemin action is depicted in Sect. V-B. The harness connectsmultiple sensors together and provides a Universal Serial Bus(USB) interface for a PC to control the setup. The depictedconfiguration contains only light and colour sensors which canbe placed over the various indicators LEDs on a target ECU. Nogeneral-purpose microcontroller is used in the harness, whichallows the entire setup to be programmed and configured fromthe PC in a higher-level language, Python in our case.
Figure 2. Sensor harness connection schema showing (1) the FT232H I2C-to-USB converter, (2) an TCA9548A I2C Multiplexer, and (3) two ISL29125light & colour sensors.
The light sensors are connected through an I2C bus, acommunication interface which is present on many low-costsensors. The use of I2C make the harness extensible withvarious other sensors, such as sound sensors to monitor auditoryalerts, motion sensors to monitor steering wheel movement,or current sensors to detect an engine start. As the I2C bussupports multiplexing, multiple sensors can be connected tothe same bus as long as they have a different I2C address. Thelow-cost sensors used in the harness often have a fixed address,which implies that an I2C multiplexer must be used to connectmultiple sensors of the same type in the harness. In order tointerface with the sensor harness from a PC an USB to I2Cadapter is used.
1) Light & Colour Sensors: ISL29125.:
The ISL29125colour sensor is used to measure the status of visual indicatorson automotive equipment. The sensor provides a simple RGBlight level readout and has a number of configuration optionswhich determine the sensitivity and precision of the sensor.The sensitivity can be configured between 10k lux or 375 luxand the sensor has a built in infrared light filter which canbe configured separately. The precision of the sensor can beconfigured to be either 12 or 16 bits. Changing the precisionalso changes the integration time, meaning a higher precision(16 bits) will require the Analogue-Digital Converter (ADC)to sample the sensor longer resulting in slower measurements.In addition to the I2C interface the sensor has an interrupt pinwhich can be triggered by a configurable light level on eitherof the red, green or blue channel. Currently the harness doesnot use the interrupt functionality but works by polling eachsensor individually. Our colour sensors have a single fixed I2Caddress, requiring a multiplexer to connect multiple sensorson the same I2C bus.
2) I2C Multiplexer: TCA9548A.:
As the colour sensor hasa fixed I2C address, the TCA9548A multiplexer is used toconnect multiple colour sensors in the harness. The TCA9548Amultiplexer has eight I2C channels which allows eight coloursensors to be connected on the same bus. As the multiplexeras a three bit configurable address multiple multiplexers canbe daisy chained allowing up to 64 I2C busses on a singleconnection.
3) I2C-to-USB: FT232H.:
In order to connect the I2C bus toa PC the FTDI FT232H adapter is used. This adapter supportsa number of different bus protocols such as UART, SPI andI2C in addition to a GPIO interface which can be used to writeto eight digital IO pins.
4) Programming, Calibration & Use.:
To obtain readingsfrom the light sensors, the Arduino library for readingISL29125 sensors was adapted. The resulting library uses theAdafruit library for communicating with the FT232H adapter.Our current sensor library exposes functions for initialising anew sensor, and for reading the sensor’s red, green and bluevalues.The initialisation function resets the sensor and configures itin RGB mode which enables all three colour channels, puts itin 10k-lux mode for bright environments and enables high IRlight adjustment. Reading a colour value is done by readingfrom the relevant device register, which returns a colour valueof either 12 or 16 bits depending on the chosen precision. Thetime the sensor needs to take a reading varies depending onthe ADC integration time which in turn depends on the chosenprecision: At 16 bit precision, each reading takes about 110 mswhile at 12 bit each reading takes only 7 ms. As the sensor’soutput registers are double-buffered, reading out these registersbetween sensing operations will result in outdated readings.In order to use multiple light sensors a library that interfaceswith the TCA9548A multiplexer was created, this libraryallows virtual I2C ports to be created for each channel ofthe multiplexer. These virtual I2C ports can then be used inthe sensor library instead of the default FT232H I2C port.In order to switch I2C channels the number of the requestedchannel is written before any commands, this adds a delaybefore every I2C command, which is negligible in comparisonwith the integration time of the light sensor and does not impactfuzzing performance.In initial experiments we use the colour light sensors to detectwhether the various indicators on an automotive dashboardare changing state, effectively converting the red, green andblue light levels into a binary input signal for the fuzzer. Asthe sensors are sensitive enough to detect (even reflected)movements behind the sensor while duct-taped to a dashboardin both sensitivity configurations (up to 375 lux and 10k lux)a simple threshold is not sufficient to distinguish state change.We devise a calibration method that involves taking a readingwhen the indicator is on and when the indicator is off. Thisresults in red, green and blue light level triples to which anynew measurement can be compared, if the new measurement iscloser to the on-value the indicator is detected as on and viceversa. This method assures that a uniform increase or decreasein ambient light does not change the detected indicator value.The method further requires a calibration with the indicator https://github.com/sparkfun/SparkFun ISL29125 Breakout ArduinoLibrary https://github.com/adafruit/Adafruit-FT232H-Breakout-PCB both on and off, which may not be feasible when the indicatortrigger is unknown. When the indicator cannot be triggeredduring calibration, a simple threshold may be used to detectthe indicator state. More elaborate calibration methods may berequired to operate the sensor harness in noisy environments.V. E VALUATION AND D ISCUSSION
We have applied our fuzzer implementation and the sensorharness to a number of case studies that include the ICSimautomotive instrument cluster simulator , a demo setup forillustrating and implementing message authentication in CANnetworks with the VulCAN [10] framework, as well as realinstrument clusters. In this section we focus on our experienceand lessons learned from the latter two case studies. Wecompare our findings with earlier manual approaches todiscover bugs and explore proprietary functionality in thesescenarios. A. Case Study 1: VulCAN
We evaluated the effectiveness of our fuzzer to find imple-mentation bugs and security vulnerabilities on a demo imple-mentation of VulCAN [10], a generic design for CAN messageauthentication. VulCAN provides efficient and AUTOSAR-compliant [9] authentication plus software component attesta-tion based on lightweight trusted computing technology. Weused the same test bench as described in [10] to test the abilitiesof the fuzzer.In brief, the demo consists of a number of ECUs withkeypads as input devices and LED displays as actuators.A distributed control application which simulates a trac-tion control system is executing on the ECUs. Applicationcomponents communicate via cryptographically authenticatedCAN messages with freshness guarantees: only messagesthat are successfully validated to be fresh and to originatefrom unmodified and integrity-protected remote componentshould ever be able to trigger output events. The applicationcommunicates only a few valid payloads at fixed intervals.Thus, deviation from expected behaviour would be easy todetect. Yet, since it is unlikely for a random or mutation-basedfuzzer to “guess” a valid payload, nonce, and authentication tagtriple, and since the system was designed with security in mind,we did not expect the security properties of the system to bebroken easily. This part of the evaluation is conducted withoutusing the sensor harness but by relying on visual observation onthe demo’s LED displays. The fuzzer is executing on desktopPC, which is connected to the demo setup via a USB to CANinterface.To our surprise, with the help of the fuzzer, we detectedand traced several unique vulnerabilities in the system in afairly short period of time. Below we focus on two particularlysubtle discoveries.The first vulnerability was discovered nearly instantaneouslyin a fuzzer configuration where messages with extended CANarbitration IDs are generated. Such messages resulted in system https://github.com/zombieCraig/ICSim tates where the injected messages could lead to actual displayoutputs, breaking the security properties of VulCAN entirely.Extended CAN IDs are not being used in the VulCAN demo,and thus, the components were not tested in environmentswhere these messages occur. Most likely, a misconfigured driverfor the CAN controller on an ECUs – “untrusted” softwarein VulCAN’s attacker model – together with an incompleterejection condition in a secure application module, allowedan attacker to arbitrarily adjust the displays of the test benchwithout having to pass authenticity checks.The second vulnerability was found within the implemen-tation of one of the authentication protocols in VulCAN,specifically VatiCAN [21]. This implementation turned outto be particularly vulnerable to denial-of-service attacks whenbeing flooded with specific traffic patterns, allowing an at-tacker to desynchronise nonces and render trusted componentsunresponsive even to dedicated re-synchronisation messages.The bug was discovered in a timespan of several minuteswhen fuzzing the test bench in a configuration where both,the fuzzer as well as an ECU, are simultaneously attemptingto send messages to a target ECU. Interestingly, due to theconfiguration error in CAN drivers described above, messageswith extended CAN IDs are effectively interpreted as broadcastmessages. Application components are thus subject to receivinga mix of fuzzer-generated payloads and authenticated messageswhich results in denial-of-service. B. Case Study 2: Instrument Clusters
In the context of automotive security research and forbuilding demos such as the VulCAN [10] setup, instrumentclusters are commonly used as easily accessible off-the-shelfcomponents with many visible indicators (speed needle, turningindicators, display, etc.), most of which can be controlledthrough CAN messages. Yet, the specific arbitration IDsand payloads to control these functions are not publiclydocumented. Literature on car hacking (e.g., [13]) suggestsmanual approaches to reverse-engineer these details, whichmay require hours or even days of try-and-error, even for askilled engineer. By using our sensor harness, we expect asubstantial speed-up of these processes, on top of being ableto largely automate the process.We have been experimenting with a number of clusters frompassenger cars and commercial vehicles. As illustrated in Fig. 3,components of our sensor harness are duct-taped to indicatorsof the cluster. The instrument cluster is connected to a desktopPC with the fuzzer via a USB to CAN interface and there areno other ECUs present on the CAN.In order to test the instrument cluster’s basic functions, wedeveloped a controller application that would send a numberof documented [22] CAN messages to the dashboard. Onlysome these control messages worked in combination with ourdashboard. We then applied our fuzzer in identify mode to filterthe traffic data for messages that trigger physical functions, thenapplied brute-force and mutation mode to explore arbitrationIDs and payloads to trigger further functionality. By brute-forcing the entire 11-bit arbitration ID fuzz-spacewith a fixed payload , most indicators LEDs inthe dashboard could be activated. Some of these indicators areswitched on by default when the instrument cluster is poweredup. These indicators could be triggered using a fixed payload of . Control of the speedometer and engine RPMneedles could also be triggered. The fuzzer takes about 30 sto enumerate the 11-bit address space using a delay of 10 msbetween messages. Running the identify method after findinga state changes takes an extra 30 s per activation in order toidentify the message responsible for an indicator activation.Using mutation-based fuzzing on any of the messagesidentified with the brute-force method, we were able to reverseengineer the semantics of most payload bits. For example,by starting from the messages that activated the left turningindicator, the fuzzer was able to not only identify the bitresponsible for triggering the indicator. It was also able toidentify the bit responsible for the right turn signal, theheadlights indicator and a number of other status LEDs.Enumerating eight payload bits takes about 20 s with a delayof 3 ms between messages. Using the identify method requiresan additional 5 s per indicator. The delay between messages iscritical when using mutation-based fuzzing. If the delay is tooshort, the next message, which will have another bit flipped,will overwrite the previous message. This will cause moreindicator activations to be missed or some indicators to noteven activate.
Omission fuzzing is not necessary for analysing this particularinstrument cluster. Yet, we heavily relied on this method methodwhen working with more modern clusters from commercialvehicles.
C. Discussion & Lessons Learned
The two case studies outlined above show that our fuzzerscan efficiently reveal undocumented functionality, intricatebugs and security vulnerabilities in ECUs connected to CANnetworks. Ultimately, our experiments provide further evidencefor fuzzing to be a useful tool in testing and reverse-engineering,which is due to the technique’s ability to cover an enormousrange of possible combinations of system inputs, in our casearbitration IDs and payloads. Many of these inputs may noteven occur in normal and benign operation, and are difficultto consider in static test cases.Above we describe two critical security vulnerabilities inan experimental system design, which are based on intricateimplementation and configuration bugs. Previously undetected,these vulnerabilities became apparent within minutes with theuse of a fuzzer, even without relying on a sensor harness.The harness could be used to reduce human interaction andto improve the duration for detecting and tracing these bugs,but non-trivial extensions of the harness would be requiredto sense the state of actuators such as the attached displays.Alternatively, the demo setup could be modified to featuresimpler actuators (i.e., individual LEDs or relays) that allowfor an easier detection of conditions that satisfy our bugoracles. We used our fuzzer, specifically the identify and replay igure 3. An automotive instrument cluster with (part of) our sensor harness attached. The cluster originates from a 2014 Seat Ibiza model. functionality, to trace bugs in source code and fix vulnerabilities.The resulting fixes are not straight-forward as they requireconsideration of rather involved network states. Our findingshighlight the need for thorough testing and verification ontop of strong cryptographic primitives and Trusted Computingtechnology when designing distributed control systems that arepotentially exposed to malicious interactions.We further described how the sensor harness in combinationwith our fuzzing techniques can be used to largely automatethe process of reverse-engineering communication protocols ofproprietary ECUs. Manually reversing a substantial subset ofthe functionality of, e.g., an instrument cluster, can easily bean effort of several days or even weeks. With our approach,this can be achieved within hours. Additional sensors (e.g.,audio, power consumption, vibration) could further extendthe harness’ abilities. We believe that our approach can beused to identify bugs and unintended functionality when beingapplied to components for which a specification is available.This specification could be integrated in a bug oracle such thatresponses outside of the specified behaviour are detected aserrors. In this context, our approach may be useful to automateactivities such as integration testing and to achieve a highinput-space coverage in these activities.Fuzzing ECUs under realistic conditions, i.e., while beingconnected to a vehicle’s CAN network with many other “noisy”ECUs, may also be feasible but requires fuzzing strategies thataim at noise reduction by exploring the effects of individualmessages or sequences of messages in different system states.In this context it may be useful to also consider CAN responsesof ECUs. We may borrow from recent approaches in anomalydetection in control systems (e.g., [19] and [20]) to defineoracle functions that detect changes in the response stream ofan individual ECU, or even to detect state change throughoutthe network. VI. R
ELATED W ORK
Fuzzing has a long history and is still actively developed, inparticular in the domain of security- and penetration testing of software systems [14], [15]. Recent work [16] elaborates on thedifficulties of employing fuzzing in embedded systems. Specif-ically for automotive systems, Smith states in [13] that, whilefuzzing can certainly be useful in discovering undocumentedservices or crashes, it is rarely useful beyond that, e.g., to findand exploit vulnerabilities. Our experience report disagreesslightly with this observation: We discovered that fuzzing ismore efficient in finding subtle vulnerabilities and configurationerrors than monitoring or reverse engineering the firmware andcommunications. Fuzzing exposes substantially more of thesystem’s unintended states than what one would be able toexplore manually, due to the sheer amount of pseudo-randommessage combinations that are generated and dispatched bythe fuzzer. This allows testers to focus on tracking downand responding to vulnerability reports instead of having tomanually probe the system. With automated oracle function,as discussed in Sect. III-A and Sect. IV, fuzzing becomes evenmore efficient. While our approach mostly relies on black-boxfuzzing where very little knowledge of the system is assumedand oracle functions must rely on system outputs rather thanobserving the system’s internal state, our approach can certainlybe combined with more advanced reverse-engineering andfirmware inspection tools. This would lead to more powerfuland also much more intricate oracle functions.Related research investigates the extent to which fuzzingcan be applied be in automotive systems [17], [18], [23]–[25].Our work aims to improve over this state of the art by notonly defining a fuzzer for CAN networks, but by developingan entire methodology that defines fuzzing objectives, oraclefunctions and fuzzing strategies, and substantially improvesthe automation of testing cyber-physical systems. We reporton experiments and lay out our experience from applying thismethodology to two realistic systems, one of which being aprototype for an automotive security system. While related workreports mixed results on the usefulness of fuzzing automotivenetworks, we judge our results as largely positive since we wereable to identify a few subtle vulnerabilities and dramaticallypeed up reverse-engineering activities.Further related research investigates the use of fuzzingto explore stateful communication protocols [26], [27], alsoin the context of cyber-physical systems such as smart-grid communications [28]. Our toolchain does currently notemploy such techniques. We believe that these could be asensible addition to the current stateless exploration approach.Specifically, these techniques may be used to brute-forceintricate CAN protocols that trigger, e.g., software updateson ECUs. VII. S
UMMARY & C
ONCLUSIONS
Automotive control networks are highly complex safety-critical and security-critical systems which have been shownto be vulnerable to adversarial interactions. In this paper wedevise a largely automated approach for fuzz-testing thesesystems. We discuss how bug oracles for automotive ECUscan be described, define a number of fuzzing strategies, anddevelop a sensor harness to allow oracle functions to detectinteresting system behaviour in an automated fashion. We haveimplemented our fuzzing approach in CaringCaribou, an open-source automotive penetration-testing toolkit, and we reporton two sets of experiments where we apply our fuzzer tofind vulnerabilities and to reverse-engineer proprietary ECUfunctions. To the best of our knowledge, our approach is thefirst to achieve a high degree of automation for these activities,and we see future applications of our fuzzing approach in,e.g., penetration testing but also in integration- and compliancetesting. While we have been focusing on CAN networks, webelieve that the approach is applicable to other types of controlnetworks and beyond the domain of automotive computing. Inthe future we will work towards a more rigorous evaluation ofour approach, following the methodology of [29].
1) Acknowledgements.:
This research is partially fundedby the Research Fund KU Leuven. This research is partiallyfunded under SErVO, “Secure and Economically ViableV2X Solutions”, by the Flemish Agentschap Innoveren &Ondernemen. We thank the developers of CaringCaribou fortheir sport and ideas, and for integrating parts of our fuzzerinto their platform. R
EFERENCES[1] M. Wolf, A. Weimerskirch, and C. Paar, “Security in automotive bussystems,” in
Workshop on Embedded Security in Cars , 2004.[2] T. Hoppe, S. Kiltz, and J. Dittmann, “Security threats to automotive CANnetworks – practical examples and selected short-term countermeasures,”in
Computer Safety, Reliability, and Security (SAFECOMP ’08) . Berlin,Heidelberg: Springer Berlin Heidelberg, 2008, pp. 235–248.[3] O. Henniger, L. Apvrille, A. Fuchs, Y. Roudier, A. Ruddle, and B. Weyl,“Security requirements for automotive on-board networks,” in , 2009, pp. 641–646.[4] K. Koscher, A. Czeskis, F. Roesner, S. Patel, T. Kohno, S. Checkoway,D. McCoy, B. Kantor, D. Anderson, H. Shacham et al. , “Experimentalsecurity analysis of a modern automobile,” in
Security and Privacy, 2010IEEE Symposium on . IEEE, 2010, pp. 447–462.[5] S. Checkoway, D. McCoy, B. Kantor, D. Anderson, H. Shacham,S. Savage, K. Koscher, A. Czeskis, F. Roesner, T. Kohno et al. ,“Comprehensive experimental analyses of automotive attack surfaces,” in
USENIX Security Symposium . San Francisco, 2011. [6] C. Miller and C. Valasek, “A survey of remote automotive attack surfaces,”
Black Hat USA , 2014.[7] ——, “Remote exploitation of an unaltered passenger vehicle,”
BlackHat USA
ACSAC ’17 . ACM, 2017, pp. 225–237.[11] S. Fr¨oschle and A. St¨uhring, “Analyzing the capabilities of the CANattacker,” in
ESORICS ’17 , ser. LNCS, vol. 10492. Heidelberg: Springer,2017, pp. 464–482.[12] A. Palanca, E. Evenchick, F. Maggi, and S. Zanero, “A stealth, selective,link-layer denial-of-service attack against automotive networks,” in
International Conference on Detection of Intrusions and Malware, andVulnerability Assessment . Springer, 2017, pp. 185–206.[13] C. Smith,
The car hacker’s handbook: a guide for the penetration tester .No Starch Press, 2016.[14] P. Oehlert, “Violating assumptions with fuzzing,”
IEEE Security &Privacy , vol. 3, no. 2, pp. 58–62, 2005.[15] V. J. M. Man`es, H. Han, C. Han, S. K. Cha, M. Egele,E. J. Schwartz, and M. Woo, “Fuzzing: Art, science, andengineering,”
CoRR , vol. abs/1812.00140, 2018. [Online]. Available:http://arxiv.org/abs/1812.00140[16] M. Muench, J. Stijohann, F. Kargl, A. Francillon, and D. Balzarotti, “Whatyou corrupt is not what you crash: Challenges in fuzzing embeddeddevices,” in
Proceedings of the Network and Distributed System SecuritySymposium (NDSS) , 2018.[17] H. Lee, K. Choi, K. Chung, J. Kim, and K. Yim, “Fuzzing can packets intoautomobiles,” in . IEEE, 2015, pp. 817–821.[18] S. Bayer, T. Enderle, D.-K. Oka, and M. Wolf, “Automotive securitytesting – the digital crash test,” in
Energy Consumption and AutonomousDriving . Springer, 2016, pp. 13–22.[19] A. Taylor, S. Leblanc, and N. Japkowicz, “Anomaly detection inautomobile control network data with long short-term memory networks,”in , 2016, pp. 130–139.[20] C. Wressnegger, A. Kellner, and K. Rieck, “Zoe: Content-based anomalydetection for industrial control systems,” in ,2018, pp. 127–138.[21] S. N¨urnberger and C. Rossow, “– vatiCAN – Vetted, authenticated CANbus,” in
Cryptographic Hardware and Embedded Systems – CHES ’16:18th International Conference, Santa Barbara, CA, USA, August 17-19,2016, Proceedings . Berlin, Heidelberg: Springer Berlin Heidelberg,2016, pp. 106–124.[22] L. Bataille, “Volkswagen can bus gaming,” URL: https://hackaday.io/project/6288-volkswagen-can-bus-gaming.[23] S. Bayer and A. Ptok, “Don’t fuss about fuzzing: Fuzzing controllers invehicular networks,” , p. 88, 2015.[24] R. Nishimura, R. Kurachi, K. Ito, T. Miyasaka, M. Yamamoto, andM. Mishima, “Implementation of the can-fd protocol in the fuzzingtool bestorm,” in . IEEE, 2016, pp. 1–6.[25] D. S. Fowler, J. Bryans, and S. Shaikh, “Automating fuzz test generationto improve the security of the controller area network.”[26] G. Banks, M. Cova, V. Felmetsger, K. Almeroth, R. Kemmerer, andG. Vigna, “Snooze: toward a stateful network protocol fuzzer,” in
International Conference on Information Security . Springer, 2006,pp. 343–358.[27] S. Gorbunov and A. Rosenbloom, “Autofuzz: Automated networkprotocol fuzzing framework,”
IJCSNS , vol. 10, no. 8, p. 239, 2010.[28] H. Dantas, Z. Erkin, C. Doerr, R. Hallie, and G. v. d. Bij, “efuzz: Afuzzer for dlms/cosem electricity meters,” in
Proceedings of the 2ndWorkshop on Smart Energy Grid Security . ACM, 2014, pp. 31–38.[29] G. Klees, A. Ruef, B. Cooper, S. Wei, and M. Hicks, “Evaluatingfuzz testing,” in