Wavelet Transform Analytics for RF-Based UAV Detection and Identification System Using Machine Learning
Olusiji Medaiyese, Martins Ezuma, Adrian P. Lauf, Ismail Guvenc
GGraphical Abstract
Wavelet Transform Analytics for RF-Based UAV Detection and Identification System UsingMachine Learning
Olusiji Medaiyese, Martins Ezuma, Adrian Lauf, Ismail Guvenc a r X i v : . [ ee ss . SP ] F e b ighlights Wavelet Transform Analytics for RF-Based UAV Detection and Identification System UsingMachine Learning
Olusiji Medaiyese, Martins Ezuma, Adrian Lauf, Ismail Guvenc• To detect the presence of radio-controlled UAVs in an environment by exploiting the RF signal em-anating from the UAV-flight controller communication under wireless interference (i.e., WiFi andBluetooth).• To explore the possibility of extracting RF fingerprints from the transient and steady state of the RFsignals for detection and identification of UAVs.• To utilize wavelet transform analytics (i.e., continuous wavelet transform and wavelet scattering trans-form) for the feature extraction where both coefficients and image-based signature are generated fortraining machine learning algorithms and convolutional neural network.• To evaluate the performance of trained models under varying signal to noise ratio. avelet Transform Analytics for RF-Based UAV Detection andIdentification System Using Machine Learning ⋆ Olusiji Medaiyese a , Martins Ezuma b , Adrian Lauf a and Ismail Guvenc b a Department of Computer Science and Engineering, University of Louisville, Louisville, Kentucky, 40292, USA b Department of Electrical Engineering North Carolina State University Raleigh, North Carolina, 27606, USA
A R T I C L E I N F O
Keywords :interferenceRF fingerprintingscattergramscalogramSqueezeNetUAVswavelet transform
A B S T R A C T
In this work, we performed a thorough comparative analysis on a radio frequency (RF)based drone detection and identification system (DDI) under wireless interference,such as WiFi and Bluetooth, by using machine learning algorithms, and a pre-trainedconvolutional neural network-based algorithm called SqueezeNet, as classifiers. InRF signal fingerprinting research, the transient and steady state of the signals can beused to extract a unique signature from an RF signal. By exploiting the RF controlsignals from unmanned aerial vehicles (UAVs) for DDI, we considered each state ofthe signals separately for feature extraction and compared the pros and cons for dronedetection and identification. Using various categories of wavelet transforms (discretewavelet transform, continuous wavelet transform, and wavelet scattering transform)for extracting features from the signals, we built different models using these fea-tures. We studied the performance of these models under different signal to noiseratio (SNR) levels. By using the wavelet scattering transform to extract signatures(scattergrams) from the steady state of the RF signals at 30 dB SNR, and using thesescattergrams to train SqueezeNet, we achieved an accuracy of 98.9 % at 10 dB SNR.
1. Introduction
Every new technology comes with both positive and negative impacts to our society. The unmannedaerial vehicles (UAVs) (commonly known as drone) technology is drastically evolving our world as its ap-plication is significantly broad. The immensity of UAV applications in different domains such as healthcare,logistics, remote sensing [1], data acquisition [2], precision agriculture [1], and environmental and disastermanagement [3] has led to the birth of new business opportunities. UAVs are no longer restricted to militaryusage, as civilians are rapidly adopting it for both commercial and noncommercial use. Over 1.7 millionUAVs are currently registered in the United States of America and more than 70 percent of the registeredUAVs are used for recreational activities while the remaining percentage is used for commercial purposes[4]. ⋆ This work has been supported in part by NASA under the Federal Award ID number NNX17AJ94A, and by NSF CNS-1939334Aerial Experimentation Research Platform for Advanced Wireless (AERPAW) project that supported the experiments at NC State. ∗ Corresponding author [email protected] (O. Medaiyese); [email protected] ( . Ezuma); [email protected] ( . Lauf); [email protected] ( . Guvenc)
ORCID (s):
OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 1 of 32 avelet transform analytics for UAV detection
The growing use of UAVs is raising both security and privacy concerns. UAVs are known to havebeen used for cybercrimes, terrorism, drug smuggling, invading privacy, and other malicious intents [5, 6].Aside from these malicious acts, there are few strict regulations on who can own or purchase a UAV. So,airspace contravention is another posing challenge of UAVs. For example, a hobbyist without the knowledgeof airspace regulations can fly a UAV in restricted airspace and is therefore violating airspace regulations.Every month, the Federal Aviation Administration (FAA) receives over 100 cases or incidents of UAVssighted in an unauthorized area in the United States [7]. These sightings are from pilots, citizens, and lawenforcement, which involve human efforts as UAV detection mechanisms are still not prevalent.UAV manufacturers have started enabling UAVs to have the geofencing capabilities. Geofencing isrestricting the ability of a UAV to fly into or take off within some predefined areas (e.g., areas marked asNo-Fly Zones) based on the GPS information. However, there is a need for a UAV detection system ingeofencing-free areas that are sensitive to security and privacy. While there are radar or velocity-speedguns to detect when a driver is speeding on highways by law enforcement agents, no UAV detectors havewide-spread availability for airspace and infrastructure monitoring. There is no automatic system in placefor law enforcement agencies to detect airspace contravention caused by UAVs. Solutions are still underdevelopment or at infant stages in development [8–10]. Having this solution will enable law enforcement totackle UAV crimes. Hence, the ever-increasing application of UAVs has made DDI research gain momentumin the domain of UAVs because of privacy and security issues [11, 12].The three mitigating steps for curbing security or safety threatening-UAVs are detection, identification,and neutralization [8]. Detection involves using sensors to capture necessary information that shows someattributes of the presence of a UAV in a vicinity. Identification, which is also called classification, involvesidentifying the target based on the data provided at the detection stage. The neutralization phase relies onthe output of the classification phase. If the classification phase identifies a target, then the alarm is raisedwith a counter-measure to bring down the UAV if necessary. Counter-measures may include jamming theRF communication between the UAV-flight controller, shooting down the UAV, and other methods [8]. It isimportant to note at this point that our paper focuses on detection and identification.Several sensing techniques have been exploited in the literature for detection. These include audio,visual imaging, thermal sensing, RADAR, and radio frequency (RF) signature detection. For the audio-based technique, most UAVs generate sounds (i.e., a humming sound) when in operation. The sound waves
OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 2 of 32avelet transform analytics for UAV detection
Table 1
Comparison of pros/cons for different UAV detection and tracking techniques.
Detection Tech-nique Pros Cons
Audio (e.g., [13–16]) It is cost-effective in implementation [17], its operation is pas-sive [8]. Low detection accuracy, difficult to estimate the maximum de-tection range [8], performance susceptible to ambient noise,high complexity in acquiring and maintaining an audio signa-ture database, it cannot be used for UAVs with noise cancel-lation [12].Visual Imaging(e.g., [18–20] Commercial-off-the-shelf solutions that can be rapidly utilized,it can be used for detecting autonomous UAVs. Illumination of the environment or property under surveillancecan degrade the performance, vulnerability to weather condi-tions, ineffectiveness for a crowded or cluttered area (i.e., line-of-sight (LOS) between the surveillance camera and the targetis essential), a high-quality lens with an ultra-high-resolutionmay be needed to detect UAV at a long-range, it has limitedcoverage for a large environment as a camera can only focuson one direction per unit time.Thermal sensing(e.g., [21]) It is less susceptible to weather fluctuation and backgroundclutter [8] UAVs have a low thermal signature, and it has limited coveragewhen a UAV is at non-line-of-sight (NLOS).Radar (e.g., [22,23]) It can be used for detecting autonomously controlled UAVs,and it is not susceptible to weather fluctuation. It induces interference on other RF bands, especially in acrowded environment [12], it is not effective for UAV detectionbecause the radar cross-section (RCS) depends on the abso-lute size of a flying object, making radar not effective, highcost of deployment, it requires line-of-sight to target for effec-tive and efficient operation, and high powered radars are notrecommended for areas with crowded human habitats becauseof their high active electromagnetic energy [17].RF (e.g., [24–27]) It is relatively inexpensive, that radio-controlled UAVs propa-gate RF signals, that its operation is stealthy, the size of theUAV does not affect its effectiveness, that it works for bothline-of-sight and non-line-of-sight, that it is not dependent onprotocol standardization [24], and that beyond using it for de-tecting the presence of a UAV, it can be exploited for theidentification of operation modes of UAV [28, 29]. It cannot be used to detect autonomously controlledUAVs.However, autonomously controlled UAVs are yet toreach maturity in the developmental process.It is not a trivialtask to use RF-based detection because UAV radios operate atthe industrial, scientific and medical (ISM) band and becausethere are several other devices (e.g., WiFi and Bluetooth) thatoperate at this same band, making it challenging to capturethe UAV’s RF signature. propagated from a given UAV when in operation are adopted as the audio fingerprint or signature of the UAV.Visual imaging techniques involve the use of video surveillance cameras to monitor a restricted area. Therotor or motor of UAVs emits heat and the thermogram of this heat energy can be used as a thermal signaturefor the UAV. This approach is called the infrared thermography approach or thermal sensing. Radar is themost commonly used mechanism for detecting flying objects and is found at every airport around the world.A radar system uses a radio transmitter to propagate radio pulses toward an object so that the object canreflect the radio waves. The reflected waves are received by the receiver and the temporal properties of thereflected radio waves can be used to determine or identify the object.Most UAVs use a radio-controlled (RC) communication system. The radio pulses of the communicationbetween the UAV and its flight controller can be intercepted and collected as an RF signature for UAVdetection. Using these RF signals as signatures is based on the premise that each UAV-flight controllercommunication has unique features that are not necessarily based on the modulation types or propagatingfrequencies but may be a result of imperfection in the communication circuitries in the devices.
OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 3 of 32avelet transform analytics for UAV detection
Table 1 provides the comparison of these detection techniques. While there are advantages and disad-vantages to each of the techniques, we adopted the RF approach because its operation is stealthy, that it canbe exploited to determine the operation mode of UAVs, and that it can detect UAVs that are in non-line-of-sight. The major challenge of RF detection approach is that other wireless devices (e.g., WiFi and Bluetooth)operate at the same frequency band as that of UAV communication band. However, we propose differentapproaches that reliably classify WiFi, Bluetooth, and UAV signals in this work by using wavelet transformtechniques for feature extraction. The contributions of this work are summarized as follows:1. We propose four different methods for extracting features or signatures from RF signals by usingcontinuous wavelet transforms and wavelet time scattering transforms. To the best of our knowledge,this is the first time the four methods proposed in this work have been used for DDI systems or RFdevice identification.2. We compared and contrast the effectiveness of using the image-based feature (scalogram and scatter-gram) over the coefficients (wavelet and scattering coefficients) based feature and the impact of a linearreduction in the dimensionality of the coefficients by using principal component analysis (PCA).3. While it is very common to use the transient state of RF signal for signature extraction in literature,this paper investigates the performance of adopting either transient or steady states of RF signal forfeature extraction under the proposed four methods for DDI systems. We demonstrated that the steadystate of an RF signal contained unique attributes, and it is resilient to low SNR when scattergram isextracted.4. We introduce the concept of transfer learning for UAV identification by using a publicly-available pre-trained network called SqueezeNet because it is an optimized neural network and a portable model interms of size in memory.The remainder of the paper is organized as follows. Section 2 provides a brief background and overviewof the related work. Section 3 describes the system modeling for an RF-based UAV detection system. Sec-tion 4 introduces the experimental setup and data capturing steps. Section 5 provides the feature extractionmethods proposed. Section 6 describes the UAV classification algorithm. In Section 7, the performanceevaluation and results are discussed. We provide conclusions and future work in Section 8.
OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 4 of 32avelet transform analytics for UAV detection
Transient stateNoise
Steady state
Figure 1:
RF signal from DJI Phantom 4 controller with various parts (i.e., transient and steady state) of thesignal labeled.
2. Background and Related Work
Fig. 1 shows the signature of a UAV controller (i.e., DJI Phantom 4 controller). The signature of an RFdevice is embedded in its RF signal. The uniqueness of signature is inherently attributed to the electroniccircuitries (e.g., diode, capacitors, resistors, filters, PCB, soldering, etc.) of the device [30]. Direct usage ofa signal can inhibit the computational and memory performance of the detection and identification system[31]. Hence, the extraction of a signature is an imperative step that must be done carefully. The propertiesof RF fingerprint extraction include [31]:• Minimal feature set to enable efficient time and space complexity• Intra-device repeatability and stability• Inter-device uniquenessSelecting the representative features of the signal and measuring the discrimination between two signals arethe challenges of signal classification especially when the signals are propagating at the same frequency [32][33].In RF fingerprint-based DDI, the first step is the detection of a signal, followed by extracting uniquefeatures (i.e., fingerprint) from the detected signal. There are two parts of a detected signal, which are thetransient and steady state. Fig. 1 shows a typical example of a captured RF signal from a UAV controller.The part of the signal is explicitly labeled in the figure. The transient state is short-lived, and it occurs duringpower on and off of devices[34]. On the other hand, a steady state is a reliable detection source. It has a longperiod compared to the transient state and it is the state that occurs between the interval of the start and end of
OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 5 of 32avelet transform analytics for UAV detection transient state[30]. In the literature, the transient or steady state of an RF signal has been exploited to extractcharacteristics such as instantaneous amplitude, phase, frequency, and energy envelope as representative ofthe RF signature.A transient signal has a unique amplitude variation that makes device distinguishable[30]. It has beenestablished that the transient part has a good performance in device identification in literature. The steadystate signal may contain unmodulated or modulated data. The variation in this data can make the steady stateof an RF signal lack unique properties that can be used as a signature. However, in [31], the authors used anon-transient state of orthogonal frequency-division multiplexing (OFDM)-based 802.11a signals for deviceidentification. Similarly, the authors in [30] used the steady state spectral features for the identification of aradio transmitter.Signal classification can be done either by instance-based classification or feature-based classification[32]. The instance-based classification computes the distances between the time series. Conversely, thefeature-based classification compares sets of extracted features from the signals. In this paper, we usefeature-based approaches for UAV detection and identification under inference like Bluetooth, and WiFiby exploiting wavelet analytics.Table 2 provides the summary of some related work on the RF detection technique, comparison basedon the feature extraction method, the part of the RF signal used for signature, performance evaluation (i.e.,based on accuracy and inference time) and whether wireless inferences are considered.In [24], an Auxiliary Classifier Wasserstein Generative Adversarial Networks (AC-WGANs) based modelfor UAV detection was proposed, which uses RF signatures from UAV radios as the input feature. Genera-tive adversarial networks (GAN), which is commonly used for image generation and processing was adaptedfor detection and multi-classification of UAVs. This is done by exploiting and improving the discriminatormodel of the GAN. The amplitude envelope is used to reduce the length of the original signal and principalcomponent analysis (PCA) is further employed to reduce the dimensionality of the RF signature for featureextraction. An accuracy of 95% was obtained using AC-WGANs. However, the authors do not specify whichpart of the signal was used.In [25], the channel state information (CSI) from the UAV RF channel was used to detect the presenceof a UAV in an environment by examining the effect of mobility, spatiality, and vibration in the CSI. Arule-based algorithm was used for detection based on the metrics of the three effects. The accuracy of the
OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 6 of 32avelet transform analytics for UAV detection
Table 2
Related work on RF based UAV Detection and Identification.
Literature Feature extraction method State of the signal Algorithms Accuracy Interference Inference timeTransient Steady[24] PCA N/A N/A AC-WGANs 95% None No[25] CSI N/A N/A rule-based algorithm 86.6% WiFi No[26] DWT, statistical features Yes No ML algorithms 96.3% None No[27] DWT, statistical features Yes No ML algorithms 98.13% Bluetooth, WiFi Yes[28] DFT N/A N/A DL 84.5% None No[35] DFT and CNN N/A N/A DL 94.6% None No[36] spectrogram Yes N/A CNN 99.5% Bluetooth, WiFi NoIn this work CWT, WST Yes Yes ML and DL algorithms 98.9% Bluetooth, WiFi Yes rule-based model was 86.6%In [28], discrete fourier transform (DFT) was applied to UAV RF signals extract the frequency compo-nents of the signals, and the magnitude of the frequency components was used to train a deep neural network(DNN) model. An accuracy of 84.5% was obtained when detecting and identifying the type of UAV. Theauthors did not consider other ISM devices that operate at the same frequency band (i.e., 2.4 GHz) as UAV-flight controller communication.Similarly, the authors in [29] used DFT to transform UAV signals to the frequency domain and 2048frequency bins were obtained. A multi-channel 1-D convolutional neural network (CNN) was employed toextract features from the bins. The feature representations were then used to train a fully connected neuralnetwork. An accuracy of 94.6% was achieved when identifying the type of UAVs. The effect of SNR onmodel performance and inference time was not considered or evaluated.In [26, 27], the authors proposed a k-nearest neighbours ( k NN) based model for detecting UAV usingRF signatures (control signals) under the presence of WiFi and Bluetooth signals. Bandwidth resolution wasused to identify UAV signals from both WiFi and Bluetooth. To classify the type of UAV, a two-level discretewavelet transform (DWT) was used to decompose from the UAV signals, and statistical estimation (such asmean, shape factor, kurtosis, variance, and so on) of detailed coefficients was to train the k NN model. Anaccuracy of 98.13% was achieved.
3. System Modeling for UAV Detection System
Fig. 2 shows the system modeling of an RF-based UAV detection system utilize for infrastructure surveil-lance. The system modeling gives an overview of our methodology. UAV controllers, WiFi, and Bluetoothdevices operate at the same frequency band. So, a signal interceptor (antenna) and oscilloscope are used
OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 7 of 32avelet transform analytics for UAV detection
Infrastructure under surveillance Bluetooth interferenceWiFi interference Antenna Oscilloscope Captured signalUAV UAV controllerUAV control signal
Preprocessing and Feature ExtractionUAV Classification Algorithm
Figure 2:
RF-based UAV detection system for infrastructure surveillance under WiFi and Bluetooth interfer-ence.The UAV controller signals is exploited for the detection of UAV. UAV controller, WiFi, and Bluetoothdevices operate at 2.4 GHz frequency band. An antenna is use to intercept RF signals from UAV controller, WiFi,and Bluetooth devices. The captured signal is processed and use for UAV detection and classification. for signal capturing at 2.4 GHz in a stealth manner. The captured RF signals (i.e., from UAV controllers,WiFi, or Bluetooth devices) which serve as the signature to associated devices are preprocessed and use forextraction of features by exploiting wavelet transform analytics. The extracted features are used to train MLand CNN algorithms for UAV detection and classification.
4. Experimental Setup and Data Capturing
We captured RF signals from two Bluetooth devices (a smartphone and smart wristwatch), two WiFirouters, and six UAV controllers (four DJI UAVs, one BeeBeerun UAV, and one 3DR UAV). The operationalfrequency of all the devices is 2.4 GHz. Table 3 shows the catalog of the devices used for the experiment.The data was collected in an outdoor setting under a controlled environment. Fig. 3 shows the outdoorexperimental setup. A 24 dBi 2.4 GHz grid parabolic antenna was used to intercept propagating RF signalsfrom the RF devices (i.e., UAV controllers, Bluetooth, and WiFi devices). The intercepted signal goes
OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 8 of 32avelet transform analytics for UAV detection
Figure 3:
Experimental setup for capturing RF signals from UAV controllers, WiFi and Bluetooth devices in anoutdoor setting.
Table 3
Catalog of RF devices used in the experiment for RF fingerprints acquisition. Under the UAV device, we use theUAV controllers from respective models.Device Make ModelUAV DJI Phantom 4InspireMatrice 600Mavic Pro 1Beebeerun FPV RC drone mini quadcopter3DR Iris FS-TH9xBluetooth Apple Iphone 6SFitBit Charge3 smartwatchWiFI Cisco Linksys E3200TP-link TL-WR940N through a 2.4 GHz bandpass filter which ensures that a frequency band at 2.4 GHz is acquired. An RF lownoise amplifier (LNA), FMAM63007, which operates from 2 GHz to 2.6 GHz with 30 dB gain is used toamplify the bandpass signal. A DC (direct current) generator is utilized to power the low noise amplifierattached to the bandpass filter. A 6 GHz bandwidth Keysight MSOS604A oscilloscope which has a samplingfrequency of 20 GSa/s collects and stores the captured RF signals from the devices.The oscilloscope has a threshold trigger for signal detection. We observed the background noise level inthe environment from the oscilloscope and calibrated the threshold above the background noise level. In thepresence of a signal, the energy level of the signal goes above the threshold and triggers the oscilloscope to
OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 9 of 32avelet transform analytics for UAV detection g [n]h [n]
2 2 y [n] d [n]a [n]
Figure 4:
Single level Haar wavelet decomposition for raw RF signal prepossessing. The raw captured signal fromRF devices is denoted as 𝑦 [ 𝑛 ] . The low pass and high pass filter are 𝑔 [ 𝑛 ] and ℎ [ 𝑛 ] respectively. The approximateand detail coefficients are denoted as 𝑎 [ 𝑛 ] and 𝑑 [ 𝑛 ] . Approximate coefficient, 𝑎 [ 𝑛 ] , is used for feature extraction. capture and store the signal detected. In the absence of a signal, the oscilloscope does not capture data. Thecaptured data becomes the raw RF signals that are preprocessed for the classification algorithm.Each captured signal consists of five million sampling points. For each device considered, 300 RF signalsare collected at 30 dB SNR. This implies that 3000 RF signals are collected from the ten devices which take20.1 GB of storage. We selected 200 RF signals from each device for the training set and the remaining100 RF signals for the test set. So, 2000 RF signals are used for training purposes, and 1000 RF signals areutilized for testing. In selecting the beginning of a device’s RF signal, we use a statistical changepoint detection algorithmin MATLAB to find abrupt changes in signal. The changepoint detector help us to determine the signalstates (i.e., transient and steady states) and to prevent the noisy part of the raw signal from corrupting ourRF signature. From here, the captured RF signal is then decomposed or preprocessed using a single levelHaar wavelet decomposition (HWD) to improve the computational efficiency of extracting features. Fig. 4shows the architecture of the HWD. The raw captured signal 𝑦 [ 𝑛 ] is passed into two parallel-connectedfilters which are low pass filter ( 𝑔 [ 𝑛 ] ) and high pass filter ( ℎ [ 𝑛 ] ) respectively. The output of each filter isdown-sampled. The output from down-sampling the outcome of the low pass filter, 𝑎 [ 𝑛 ] , are called theapproximation coefficients which represents the low-frequency component of the raw signal. Similarly, theoutputs from down-sampling the results of the high pass filter, 𝑑 [ 𝑛 ] , are called the detail coefficients whichis the high-frequency component of the signal. Both the transient and steady states of the signal are acquiredfrom the approximate coefficients, 𝑎 [ 𝑛 ] , of the decomposed signal. Features or RF fingerprints are acquiredfrom these states for classification. OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 10 of 32avelet transform analytics for UAV detection
5. Wavelet Analytics-based Feature Extraction
Time-frequency domain analysis is an essential step in extracting unique attributes (signatures) from asignal. The Fourier transform (FT) is the most applied mathematical function for analyzing the frequencycontent of a signal in signal processing. It essentially represents a signal, in the time domain, by series ofsines and cosines in what is called the frequency domain, revealing several important features of the signalthat were hidden or unknown in the time domain. While the FT shows the frequency of the sample, itdoes not give the frequency variation with time and it is of limited application particularly with signals ofvarying frequencies such as non-stationary signals. The Short-Time Fourier Transform (STFT) overcomesthis limitation by sliding a window through the signal at a short interval/time, along the time axis, andperforming FT on the data within that box.The outcome of STFT is essentially a decomposition of the signal in the time domain into a time-frequency representation, which gives the frequency variation of the signal over time. The effectivenessof the STFT depends on the choice of window function used. The challenge with STFT is finding a suitablewindow size that balances time resolution with frequency resolution- choosing a window that gives higherfrequency resolution gives a lower time resolution and vice versa.The Wavelet Transform (WT) is one that seeks to mitigate the challenges of STFT. Unlike the STFTwhich slides a fixed window through the signal, the wavelet transform utilizes a variable window function[37]. The wavelet transform is the convolution of a base wavelet function with the signal under analysisor consideration [38]. Wavelet transforms enable the transformation of a signal in the time domain into adomain that will allow us to analytically examine hidden properties or features that describe the signal.The WT of signal 𝑓 ( 𝑡 ) is given as [39] 𝑤 ( 𝑠, 𝜏 ) = 1 √ 𝑠 ∫ ∞−∞ 𝑓 ( 𝑡 ) 𝜓 ( 𝑡 − 𝜏𝑠 ) 𝑑𝑡, (1)where 𝑠 > is scaling factor, 𝜓 ( 𝑡 − 𝜏𝑠 ) is the template function-based wavelet and 𝜏 is the time shiftingfactor. The wavelet transforms takes a signal and decomposes it into a set of basis functions [40]. Thesebasis functions are obtained by using a single template function (base wavelet) to perform both scaling andshifting operations along the time axis. Scaling is the process of stretching or shrinking the wavelet tomatch the feature of interest. The scaling factor, 𝑠 , is used to categorize this process. The higher the 𝑠 , the OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 11 of 32avelet transform analytics for UAV detection
Decomposed signal( a[n] )CWT WSTMean of Wavelet Coefficients Mean of Scattering Coefficients
Scalogram Scattergram
Figure 5:
Divisions of the four proposed feature extraction methods based on CWT and WST. The decomposedsignal, 𝑎 [ 𝑛 ] , is the approximation coefficient from the signal HWD. higher the stretch, and the lower the 𝑠 the higher the shrink. The scaling factor is inversely proportional tothe frequency of the signal [40]. The shrinking factor, 𝜏 , is used to move the signal along the time axis. Inwavelet transform, the similarities between the signal and a template function-based wavelet can be extractedduring the decomposition process. These similarities are in a form of wavelet coefficients [39] and they canrepresent underlying features or attributes in the signal.We proposed four different methods that are based on wavelet transforms which are grouped into contin-uous wavelet transform (CWT) and wavelet scattering transform (WST). Fig. 5 shows the categories of thefeature extraction methods. The approximate coefficients, 𝑎 [ 𝑛 ] , of the raw signal are used to extract signa-tures by using wavelet transforms. We compare and contrast the performance of these four feature extractiontechniques. Continuous wavelet transform (CWT) is defined as in (1). A derivative of STFT is the spectrogram,which is also used for analyzing signals in the time-frequency domain and it is the squared value of theSTFT that provides the energy distribution in the signal in the time-frequency domain [39]. Similarly, CWTprovides a wavelet spectrogram which gives the distribution of energy in the signal in the time-scale domain.The wavelet spectrogram, also known as scalogram, is the squared modulus of the CWT or a plot of the
OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 12 of 32avelet transform analytics for UAV detection F r e qu e n cy ( G H z ) (a) F r e qu e n cy ( G H z ) (b) F r e qu e n cy ( G H z ) (c) Figure 6:
Examples of scalogram extracted from the transient state of the captured RF signals: (a) IPhone 6S,(b) DJI Inspire, and (c) TPLink (WiFi device). F r e qu e n cy ( G H z ) (a) F r e qu e n cy ( G H z ) (b) F r e qu e n cy ( G H z ) (c) Figure 7:
Examples of scalogram extracted from the steady state of the captured RF signals: (a) IPhone 6S,(b) DJI Inspire, and (c) TPLink (WiFi device). energy density 𝐸 ( 𝑠, 𝜏 ) [39]. 𝐸 ( 𝑠, 𝜏 ) = | 𝑤 ( 𝑠, 𝜏 ) | . (2)The scalogram of a signal 𝑥 ( 𝑡 ) can serve as the signature for the signal. Morlet, Mexican hat, Gaussian,frequency B-Spline, harmonics, and Shannon wavelets are examples of base wavelets commonly used forCWT [38, 40].The implementation of CWT by continuously varying the scale parameter, 𝑠 , and translation parameter, 𝜏 , introduces redundant information that may not be of value to the specific application [40]. To eliminatethis redundancy, the scale and translation parameters are discretized leading to the discrete wavelet transform[38, 40]Two categories of features are extracted from the CWT of the signal. These are the transform’s coeffi-cients and the scalogram (i.e., an image). We transformed our signals using the implementation of the CWTfunction in MATLAB. The real part of the transform’s coefficients is averaged and transposed as a feature OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 13 of 32avelet transform analytics for UAV detection
Feature
Extraction
Convolution (Wavelets ψ ) f * ψ Nonlinearity (Modulus) | f * ψ | Averaging (Scaling Function) | f * ψ | * ϕ Figure 8:
Wavelet scattering transform process flow with the three successive steps in transforming a signal. set for the signal representation. As a result of the transformation and averaging, a set of 114 features isacquired as the signature for each signal and this feature set is used to train the ML algorithms. Similarly,we generated the scalogram of the signal as a signature from CWT which is then used to train CNN basedalgorithm (SqueezeNet). Fig. 6 and Fig. 7 show the examples of some scalograms generated from a UAVcontroller (i.e.,DJI Inspire), WiFi and Bluetooth devices with respect to the transient and steady states of theRF signal.
WST is an improved time-frequency analytical technique for signals which are based on the wavelettransform. It allows the derivation of low-variance features from signals which can be used by a classifier todiscriminate signals [41–44].The three key factors of WST are [41]:• invariance or insensitivity to translation of signals,• stability to signal deformation,• discriminative signals (i.e., informative feature representation).The moving average of the signal enhances invariance and stability. On the other hand, the modulus ofa wavelet transform is stable and discriminative. WST exploits these two concepts for an improved time-frequency analysis. The framework of WST is synonymous with the CNN architecture for feature extractionbut the computational process does not involve the learning of parameters [41]. WST involves three succes-sive processing of signals as shown in Fig. 8. These are the convolution of the wavelet function with a signal,non-linearity by applying modulus operation and low pass filtering averaging using a scaling function.The wavelet function in wavelet scattering transform is a dilated mother wavelet 𝜓 with a scaling factor − 𝑗 , where 𝑗 varies from 1 to 𝐽 (i.e., maximum scattering level) as follows: 𝜓 𝑗,𝑘 ( 𝑢 ) = 2 −2 𝑗 𝜓 (2 − 𝑗 𝑢 ) . (3) OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 14 of 32avelet transform analytics for UAV detection f | f * ψ | | f * ψ | | f * ψ | | f * ψ | * ϕ J | f * ψ | * ϕ J | f * ψ | * ϕ J || f * ψ | * ψ | || f * ψ | * ψ | || f * ψ | * ψ | || f * ψ | * ψ | * ϕ J || f * ψ | * ψ |* ϕ J || f * ψ | * ψ |* ϕ J || f * ψ | * ψ |* ϕ J f * ϕ J Scattering levelj = 0 j = 1 j = 2
Scattering coefficients
Figure 9:
Tree of wavelet scattering transform algorithm for signal decomposition with mathematically expressionsfor computing the scalogram and scattering coefficients at each scattering levels.
Fig. 9 shows the tree algorithm for the WST framework. The node of the tree contains the scalogram co-efficients while the scattering coefficients are the convolution of the scalogram coefficient and the scalingfunction. The first step in WST is to get the translation invariance coefficients by averaging the input signal 𝑓 with a low pass filter (scaling function) 𝜙 𝐽 . These translation invariance coefficients are the zeroth-orderscattering coefficients and it is denoted as 𝑆 : 𝑆 = 𝑓 ∗ 𝜙 𝐽 . (4)The subsequent steps involve the convolution of the input signal with each wavelet filter 𝑘 in the first filterbank, followed by the non-linearity operation. This is accomplished by taking the modulus of each of thefiltered outputs. This results in the nodes at the first level (i.e., 𝑗 = 1 ) which are the scalogram for the firstlevel, 𝑈 , as: 𝑈 = | 𝑓 ∗ 𝜓 ,𝑘 | . (5)The averaging of each modulus with the scaling function will yield the first-order scattering coefficients, 𝑆 , OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 15 of 32avelet transform analytics for UAV detection as follows: 𝑆 = | 𝑓 ∗ 𝜓 ,𝑘 | ∗ 𝜙 𝐽 . (6)This iterative process is carried out on the next 𝑗𝑡ℎ level using each wavelet filter 𝑘 in the 𝑗𝑡ℎ filter bank.Similar to CWT, we proposed two feature extraction methods using WST which are the coefficient basedfeature set and image-based signature (scattergram). The WST framework uses the Gabor (analytic Morlet)wavelet. The Gabor wavelets provide an optimal resolution in both time and frequency domains whichoptimally extract local features in signals or images [42, 45]. The implementation of the WST algorithm inMATLAB R2020b is adopted for this work. In WST, as we iterate through the scattering levels, energy isdissipated so two scattering levels are sufficient for systems that use the framework [42]. This is becauseexperimental results in literature have shown that the third level scattering coefficients can have energybelow one percent [42]. Hence, we propose a two wavelet filter bank for our WST framework. Fig. 10(a)and Fig. 10(b) show the Gabor wavelets in the first and the second filter bank respectively which are used tocompute the scattering coefficients at each level. The quality factors for the first and the second filter bankare eight and four, respectively.We average each scattering coefficient generated by the WST framework and this resolved to a total of1376 features set for each signal. This feature set is used to train the ML algorithm for classification purposes.More so, for the image-based signature, we extracted the scattergram of the scalogram coefficients from thefirst filter bank to train a pre-trained CNN model called SqueezeNet. Fig. 11 and Fig. 12 depict the examplesof some scattergrams generated from a UAV controller (i.e.,DJI Mavic Pro 1), WiFi and Bluetooth devicesbased on state of the RF signal used for feature extraction. OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 16 of 32avelet transform analytics for UAV detection (a) (b)
Figure 10:
Filter bank for the WST framework: (a) First filter bank, and (b) Second filter bank. (a) (b) (c)
Figure 11:
Examples of scattergram extracted from the transient state the captured RF signals: (a) IPhone 6S,(b) DJI Mavic Pro 1, and (c) TPLink (WiFi device). (a) (b) (c)
Figure 12:
Examples of scattergram extracted from the steady state the captured RF signals: (a) IPhone 6S,(b) DJI Mavic Pro 1, and (c) TPLink (WiFi device).
6. UAV Classification Algorithm
Most of the DDIs system in the literature use statistical and ML techniques to make data-driven detectionor classification. ML and DNN (also known as deep learning) are widely used in pattern recognition and
OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 17 of 32avelet transform analytics for UAV detection
Training Data (a[n]) Feature Extraction
Machine
Learning
Algorithm
Model
Test Data (a[n])
Feature
Extraction Signal TypeTrain Data (a[n]) Feature Extraction
Machine
Learning
Algorithm
TestingTraining
Figure 13:
A flowchart of the machine learning model. provide a means of classifying UAVs based on the patterns in their signatures. ML and DNN algorithmsare tools that help in finding a mathematical function or rule-based function that can mimic a given systemusing n-dimensional input space data, where n is the number of features.Given 𝑦 = 𝐹 ( 𝑥 ) from a training set 𝑇 = ( ⃗𝑥 , 𝑦 ) , ( ⃗𝑥 , 𝑦 ) , ( ⃗𝑥 , 𝑦 ) , . . . , ( ⃗𝑥 𝑚 , 𝑦 𝑚 ) where ⃗𝑥 𝑖 is feature vectorsand y is the label. If 𝑦 ∈ 𝐶 ; and where 𝐶 is a finite space, then determining 𝑦 is a classification problem.Then we can derive the function 𝐹 ( 𝑥 ) as a model from the training set 𝑇 with 𝑚 number of samples to mapfuture feature vectors.The DDI problem can be modeled as a classification problem where the RF signature of a UAV is used foridentifying the UAV. Classical ML algorithms such as k NN, SVM and ensemble are used for classificationin this work. We also use SqueezeNet which is a CNN based algorithm for classification.SqueezeNet is a pre-trained model with a small CNN architecture designed to reduce the number ofparameters [46]. It has 18 layers and the model can classify images into 1000 classes (i.e., animals, pencil,mouse, etc.) Rather than rebuilding a model from scratch, the concept of transfer learning [47] was adoptedby using SqueezeNet because the network has learned diverse feature representations from over 1000 images.The architecture of the algorithm was configured for the number of classes in our data set.The ML modeling aspect of this work follows the flowchart in Fig 13. The training data (captured RFsignal) is first decomposed using a single level HWD as discussed in Section 4.2 and the approximationcoefficient 𝑎 [ 𝑛 ] is used for extracting the corresponding RF signature or feature sets of the signal. After thefeature extraction, the extracted features are used to train an ML algorithm. Similarly, the raw test data isdecomposed to compute the approximation coefficients, and using the same feature extraction technique asthe training set, the extracted feature set is passed into a trained model for classification of signal type.Three classical ML algorithms are used to train coefficient based signatures in this work. These areSVM, k NN and ensemble. Because the dimensionality of the coefficient based features is quite large (i.e.,
OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 18 of 32avelet transform analytics for UAV detection
CWT has 114 features and WST has 1376 features), we apply principal component analysis (PCA) to reducethe dimensionality. On the other hand, SqueezeNet was used to train image-based signatures.The training of the models is done in MATLAB R2020b using the classification learner and deep net-work designer frameworks for both ML algorithms and SqueezeNet respectively. We experimented on theuniversity of Louisville Cybersecurity Education data center where a Common KVM 4 Core system with300 GB RAM and 120 GB storage was used.Rather than using the grid search technique which is computationally tedious for hyperparameter tuningor randomized search which is a random process, Bayesian optimization is adopted for the hyperparametertuning of the ML algorithms to obtain the best hyperparameters for the classifiers. This optimization methodis based on the "Bayes’ theorem" which takes advantage of the prior observations of the classifier’s loss, todetermine the next optimal hyperparameters that will reduce the loss [48].
7. Experimental Results and Discussions
We present the results of each trained model by using the test data to evaluate the performance of themodel. We focus our results on seven main directions by comparing and contrasting the results of the fourproposed feature extraction approaches. These directions are:• the state of the RF signal used for feature extraction;• the behavior of the model when the classification is a three-class problem (i.e., classifying the signalsto Bluetooth, WiFi, and UAV signal) and when the model is a ten class problem (i.e., identifying eachdevice);• the use of coefficient based signature over an image-based signature;• the impact of reducing the dimensionality of coefficient based signature on model performance;• model performance under varying SNR;• computational time complexity of feature extraction methods and the classifier’s inference time;• Lastly, the performance of the type of waveform transforms used for feature extraction.In a classification problem, there are several metrics for evaluating the performance of a classifier. Theseinclude accuracy, precision, recall, and 𝐹 -score. Using accuracy can misrepresent the performance of a OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 19 of 32avelet transform analytics for UAV detection classifier especially when the test data is skewed, so a confusion matrix is used in addition. Because ofspace limitation, we presented some of the confusion matrices of our models, and also, where the confusionmatrix is not shown, we provided the precision, recall, and 𝐹 -score of the model, which give the summaryof the confusion matrix of the model for a thorough understanding of the model performance. The definitionof these metrics is given belowAccuracy = 𝑇 P + 𝑇 N 𝑇 P + 𝑇 N + 𝐹 P + 𝐹 N , (7)Precision = 𝑇 P 𝑇 P + 𝐹 P , (8)Recall = 𝑇 𝑃 𝑇 𝑃 + 𝐹 𝑁 , (9) 𝐹 score = 2 ( 𝑃 𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 × 𝑅𝑒𝑐𝑎𝑙𝑙𝑃 𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 + 𝑅𝑒𝑐𝑎𝑙𝑙 ) , (10)where 𝑇 P , 𝑇 N , 𝐹 P and 𝐹 N represent true positive, true negative, false positive and false negative, respectively. Table 4 shows the average accuracy of the classifiers using the proposed feature extraction methods underthe two signal states (i.e, transient and steady states) at 30 dB SNR. While we focus on the classificationaccuracy as the main metric, we provided recall, precision, and 𝐹 -score values for the classifiers in Table 5,Table 6, and Table 7, respectively, as additional information to show that the classifiers generalize well andthat there are no forms of bias due to unbalanced data (i.e., in group-device classification) in our classificationaccuracy. For any given classifier, the four performance metrics are within the same range.From Table 4, using the coefficients of CWT of the signal irrespective of the signal states, in determining OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 20 of 32avelet transform analytics for UAV detection the three types of device (UAV controller, Bluetooth, and WiFi), the ML algorithms and SqueezeNet yield anaverage classification accuracy of over . k NN outperforms other ML algorithms (i.e, SVM and ensem-ble) and SqueezeNet with an average classification accuracy of . . This shows that either of the stateshas sufficient information in classifying the device type. Because the row-wise averaging of the coefficientsof CWT for each signal resulted in a 114 feature set and overloading of data through high dimensionalitycould restrain the performance of a model [49]. Hence, in an attempt to remove the redundant feature toavoid model overfitting we use PCA for dimensionality reduction of the feature set using explainedvariance as a constraint. This reduced the feature set from 114 to 1. Using one principal component as afeature set by exploiting the transient state of the signal, the accuracy of the ML models falls to . , . and . for k NN, SVM, and ensemble, respectively. Likewise, by exploiting the steady state of the signal,a significant drop in accuracy was seen when using the PCA on the CWT coefficients. The accuracy of k NN, SVM, and ensemble are , . , and . , respectively. k NN outperforms other ML algorithmsbefore applying PCA but it ended with the lowest average accuracy after the dimensionality reduction whenexploiting either the transient or steady state. Squeezenet which uses the image-based signature (scalogram)gives an accuracy of and . for the transient and steady state respectively.When CWT is used for the identification of each device (i.e., the 10 classes problem) the accuracies ofthe models are reduced and k NN outperforms other ML algorithms using the steady state of the signals withan accuracy of . The performance of k NN decreased the most when the feature space (i.e., coefficientsof CWT) is reduced using PCA to about . . The performance of the ML models significantly declineswhenever PCA is applied to the coefficients of CWT irrespective of the signal state or number of classes.On the other hand, the SqueezeNet gives average accuracy of . and . for the transient and steadystate respectively.For the WST based feature extraction method, averaging each set of scattering coefficients generated fromthe signal transform resolved to a 1376 feature set. These feature sets are used to training the ML algorithm.Similarly to CWT, we apply PCA to reduce the feature dimensionality with explained variance. Thenumber of principal components (PCs) that accounted for the explained variance varies depending onthe state of the signal. When transforming the transient state of the signal with WST and reducing the featuresets with PCA, it reduces the feature set from 1376 to 131. Conversely, the feature set was reduced from1376 to 71 using the steady state for feature extraction. OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 21 of 32avelet transform analytics for UAV detection
Table 4
Average classification accuracy (%) of the ML models and SqueezeNet based on thefeature extraction methods and the state of RF signal use for fingerprinting at 30 dBSNR.s/n Algorithm Continuous Wavelet Transform Wavelet Scattering Transform3 Classes 10 Classes 3 Classes 10 ClassesTransient Steady Transient Steady Transient Steady Transient Steady1 KNN 99.8 99.7 83.8 87.0 99.8 99.9 87.9 87.32 SVM 99.5 99.1 81.8 80.8 99.6 99.9 88.2 87.43 Ensemble 99.5 99.4 85.2 84.9 99.9 99.9 90.9 85.04 KNN + PCA 92.5 79.0 68.8 57.1 99.9 99.8 88.3 87.25 SVM + PCA 92.9 79.3 69.6 57.6 99.7 99.7 89.3 89.46 Ensemble + PCA 92.9 79.1 70.0 60.7 98.9 99.7 87.4 85.97 SqueezeNet 99.0 99.4 88.4 77.4 99.2 99.5 88.5 76.1
Table 5
Average classification recall (%) of the ML models and SqueezeNet based on the featureextraction methods and the state of RF signal use for fingerprinting at 30 dB SNR.s/n Algorithm Continuous Wavelet Transform Wavelet Scattering Transform3 Classes 10 Classes 3 Classes 10 ClassesTransient Steady Transient Steady Transient Steady Transient Steady1 KNN 99.7 99.5 84.3 87.3 99.8 99.8 88.1 87.72 SVM 99.3 98.8 84.2 81.6 99.4 99.8 88.6 88.73 Ensemble 99.3 99.4 86.8 86.3 99.9 99.8 91.5 86.54 KNN + PCA 89.5 72.5 71.6 55.9 99.9 99.8 88.9 88.05 SVM + PCA 90.1 73.0 72.1 57.1 99.5 99.6 89.8 89.96 Ensemble + PCA 89.9 72.6 73.1 60.3 98.9 99.7 88.1 86.87 SqueezeNet 98.5 99.0 88.8 76.5 99.0 99.2 88.7 77.2
Table 6
Average classification precision (%) of the ML models and SqueezeNet based on thefeature extraction methods and the state of RF signal use for fingerprinting at 30 dBSNR.s/n Algorithm Continuous Wavelet Transform Wavelet Scattering Transform3 Classes 10 Classes 3 Classes 10 ClassesTransient Steady Transient Steady Transient Steady Transient Steady1 KNN 99.9 99.8 83.8 87 99.8 99.9 87.9 87.32 SVM 99.5 99.3 81.8 80.8 99.6 99.9 88.2 87.43 Ensemble 99.4 99.1 85.2 84.9 99.8 99.9 90.9 85.04 KNN + PCA 90.9 74.1 68.8 57.1 99.8 99.8 88.3 87.25 SVM + PCA 90.8 76.8 69.6 57.6 99.7 99.7 88.3 89.46 Ensemble + PCA 91.5 75.4 70.0 60.7 98.6 99.6 87.4 85.97 SqueezeNet 99.3 99.7 88.4 77.4 99.0 99.7 88.5 76.1
The WST-based feature extraction method exhibits better performance when compared with CWT basedfeature extraction method. The accuracy of models trained with WST-based coefficients(i.e., scattergramcoefficients) for the three-class problem (i.e., group-device classification) is . , . , and . for k NN, SVM, and ensemble, respectively, when utilizing the transient state of the signal for feature extraction.
OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 22 of 32avelet transform analytics for UAV detection
Table 7
Average 𝐹 -score (%) of the ML models and SqueezeNet based on the feature extractionmethods and the state of RF signal use for fingerprinting at 30 dB SNR.s/n Algorithm Continuous Wavelet Transform Wavelet Scattering Transform3 Classes 10 Classes 3 Classes 10 ClassesTransient Steady Transient Steady Transient Steady Transient Steady1 KNN 99.8 99.7 83.6 86.9 99.8 99.9 88.0 87.42 SVM 99.4 99.0 82.5 81.1 99.5 99.9 88.4 87.83 Ensemble 99.3 99.3 85.5 85.3 99.9 99.9 91.0 85.34 KNN + PCA 90.1 72.7 68.3 55.3 99.9 99.8 88.5 87.35 SVM + PCA 90.5 74.4 69.7 55.7 99.6 99.7 89.5 89.66 Ensemble + PCA 90.5 73.6 69.8 59.5 98.8 99.7 87.7 86.27 SqueezeNet 98.9 99.3 88.5 76.3 99.0 99.4 88.6 76.0 Applying PCA to scattering coefficients of the transient state of the signals does not affect the ML classifiersnegatively as compared to CWT coefficients. k NN and SVM classification accuracies increased by . afterthe scattering coefficients reduction by PCA when exploiting the transient of the signals. On the other hand,the ensemble classifier’s average accuracy is reduced by . . More so, using the scattering coefficients ofthe steady state outperform using the CWT coefficients based on the average classification accuracy of theML algorithms and SqueezeNet.Furthermore, using WST-based features for specific-devices classification (i.e., 10 classes), the perfor-mance of the ML algorithms and SqueezeNet for both transient and steady state outperforms CWT-basedfeatures. For instance, the average accuracy for k NN, SVM, ensemble, and SqueezeNet when exploitingthe transient of the signals are , . , , and . , respectively. More so, there is no significantnegative impact on the performance of the ML classifiers when PCA is used to reduce the dimensionality ofscattergram coefficients. For example, the performance of k NN and SVM classifiers improves after featurereduction to . and . respectively using the transient state.We observed that the classification accuracy of group-device classifiers is higher than the specific-devicesclassifiers. This is because the misclassification rate is higher among UAVs from the same manufacturer (i.e.,the DJI UAVs). Fig. 14(b) and Fig. 14(d) show examples of two confusion matrices for specific-devicesclassifiers (i.e., SqueezeNet classifier where the transient state and CWT framework are used and Ensemble+ PCA classifier where the transient state and WST framework are utilized). In Fig. 14(b), the SqueezeNetclassifier misclassifies of DJI Inspire signals as DJI Matrice 600 (i.e., DJI M600) and vice-versa. Onthe other hand, the Beebeerun and 3DR Iris FSTH9X which are UAVs from different manufacturers are and correctly classified respectively. Similarly, in Fig. 14(d), it can be seen that the errors cluster around OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 23 of 32avelet transform analytics for UAV detection (a) (b) (c) (d)
Figure 14:
Confusion matrices: (a) Group-device classification using the transient state, CWT framework and k NN, (b) Specific-device classification using transient state, CWT framework and SqueezeNet (c) Group-deviceclassification using the steady state, WST framework and SqueezeNet, and (d) Specific-device classification usingtransient state, WST framework and Ensemble + PCA. the DJI UAVs.
Due to the nature of RF signals, accuracy or other performance metrics of a classifier is not sufficientin evaluating the proposed classifiers. The operation environment of an RF device affects the SNR of anRF signal. Because we built our classifiers using signals captured at 30 dB SNR, it is important to evaluatethe performance of the classifiers at different SNR levels. Adding Additive White Gaussian Noise (AWGN)to signals is a common mechanism for varying the SNR of a signal. By varying the SNR of the signals,
OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 24 of 32avelet transform analytics for UAV detection
SNR (dB) A cc u r a cy ( % ) KNNSVMEnsembleKNN+PCASVM+PCAEnsemble+PCASqueezeNet (a)
SNR (dB) A cc u r a cy ( % ) KNNSVMEnsembleKNN+PCASVM+PCAEnsemble+PCASqueezeNet (b)
SNR (dB) A cc u r a cy ( % ) KNNSVMEnsembleKNN+PCASVM+PCAEnsemble+PCASqueezeNet (c)
SNR (dB) A cc u r a cy ( % ) KNNSVMEnsembleKNN+PCASVM+PCAEnsemble+PCASqueezeNet (d)
Figure 15:
SNR results: (a) Group-device classification using transient state and CWT framework, (b) Group-device classification using steady state and CWT framework, (c) Specific-device classification using transient stateand CWT framework, and (d) Specific-device classification using steady state and CWT framework. the behavior of our classifiers is studied. Fig. 15 and Fig. 16 show the performance of our classifiers undervarying SNR based on the feature extraction methods (i.e., CWT and WST), the state of signal used for theextraction of features, and the number of classes, respectively.Fig. 15(a) shows the performance of the group-device classifiers when CWT is used for extracting fea-tures from the transient state of the RF signals. One important factor to note about the test data for group-device classifiers is that it is unbalanced data. That is, of the test data includes UAV signals, whileBluetooth and WiFi signals constitute each. In some cases, when the SNR is below 30 dB and the aver-age accuracy of a group-device classifier is , the model classifies all signals as a UAV controller signals.
OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 25 of 32avelet transform analytics for UAV detection
SNR (dB) A cc u r a cy ( % ) KNNSVMEnsembleKNN+PCASVM+PCAEnsemble+PCASqueezeNet (a)
SNR (dB) A cc u r a cy ( % ) KNNSVMEnsembleKNN+PCASVM+PCAEnsemble+PCASqueezeNet (b)
SNR (dB) A cc u r a cy ( % ) KNNSVMEnsembleKNN+PCASVM+PCAEnsemble+PCASqueezeNet (c)
SNR (dB) A cc u r a cy ( % ) KNNSVMEnsembleKNN+PCASVM+PCAEnsemble+PCASqueezeNet (d)
Figure 16:
SNR results: (a) Group-device classification using transient state and WST framework, (b) Group-device classification using steady state and WST framework, (c) Specific-device classification using transient stateand WST framework, and (d) Specific-device classification using steady state and WST framework.
In this instance, the precision and recall of the model are . and , respectively. For example, inFig. 15(a), SVM gives an accuracy of from 25 dB to 0 dB. In this instance, any signal that goes throughthe model is classified as a UAV controller signal. Despite that, the accuracy of the model is . at the30 dB SNR. When this case occurs we call it a random classification. All the classifiers with coefficientbased signatures are equal to or below the threshold of 60 accuracy from an SNR 5 dB and below, as shownin in Fig. 15(a).Conversely, in exploiting the steady state of RF signals using CWT for feature extraction under thegroup-device classification as shown in Fig. 15(b)., ensemble + PCA, k NN + PCA, and SVM + PCA make
OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 26 of 32avelet transform analytics for UAV detection non-random classification from 30 dB to 5 dB and there is a steady decrement in the accuracy as the SNRdecreases. Similarly, Squeezenet which uses scalogram joined the progression of non-random classificationat 10 dB, and the performance of SqueezeNet, in this case, decreases with a decrease in the SNR levels.SqueezeNet outperforms other classifiers at 30 dB to 10 dB SNR as shown in Fig. 15(a) and Fig. 15(b).Fig. 15(c) shows the performance of specific-device classifiers under varying SNR when CWT is usedfor signature extraction on the transient state of the signals. Squeezenet outperforms other ML algorithmsin terms of responsiveness to varying the SNR. From 0 dB SNR and above, SqueezeNet has higher accuracycompared to other ML algorithms. Similar performance is observed for SqueezeNet from 25 dB to 5 dBwhen using CWT on the steady state of the signal for specific-device classifiers shown in Fig. 15(d) Also, inFig. 15(c) and Fig. 15(d), it can be seen that the ML algorithms and SqueezeNet accuracies are decreasingwith decrease in the SNR.Fig. 16(b) shows the performance of the group-device classifiers where WST is used for extracting fea-tures from the steady state of the RF signals. SqueezeNet gives an accuracy that is above at an SNR of0 dB and this continues to increase as the SNR increases. At 10 dB, the accuracy stands as high as . .This outperforms the scenario when WST is used on the transient state of the signal or CWT for featureextraction as shown in Fig. 15(a) and Fig. 15(b) for group device classification.Fig. 16(c) and Fig. 16(d) depict a specific-device classifier where WST is used for feature extraction onthe transient and steady state of the signal, respectively. SqueezeNet also outperforms the ML algorithmsfrom 25 dB to 0 dB. We considered the computational running time for each feature extraction method and the inference timefor each classifier. Fig. 17 shows the average running time for extracting each category of features. Extractingthe image-based features (scalogram and scattergram) from RF signals have higher average running timeswhen compared with the coefficient-based features (wavelet and scattering coefficients). It can be seenin Fig. 17 that the two image-based features (i.e., scalogram and scattergram) have the highest time cost.Extracting scalogram from the CWT framework has the highest running time. Scattergram has the second-highest running time and it is immediately followed by the scattering coefficient-based method. On the otherhand, the wavelet coefficients from the CWT framework has the lowest running time.Depending on the type of feature used by a classifier, the end-to-end computational time, from the point
OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 27 of 32avelet transform analytics for UAV detection
ScalogramScattergramScattering CoefficientsWavelet Coefficients F ea t u r e E x t r a c t i on M e t hod Running Time (seconds)
Figure 17:
Running time for the feature extraction methods. a signal is captured to inference, is the sum of run time for extracting feature and inference time. Table8 shows the inference time of the classifiers. From Table 8, the average inference time of the scatteringcoefficient-based (i.e., using the coefficient of WST as input features) classifiers is higher than the classifiersthat take in CWT coefficients as input features. For instance, average inference time for k NN, SVM, andensemble classifiers under group-device classification using the transient state (i.e., three classes where CWTcoefficients are the input feature) are millisecond, . millisecond, and . millisecond, respectively.On the other hand, using scattering coefficient-based features from the transient state, the average inferencetime for k NN, SVM, and ensemble classifiers under group-device classification are millisecond, millisecond, and millisecond, respectively. This is primarily due to the higher dimensionality of thescattering coefficient-based features (i.e., 1376 feature set) as compared to wavelet coefficient-based features(i.e., 114 feature set).Furthermore, the average inference time of SqueezeNet when using scalogram is higher than when usingscattergram. For example, when the scalogram from the transient state of an RF signal is used for group-device classification, the inference time is millisecond. On the other hand, with scattergram, it takes millisecond.Overall, exploiting wavelet coefficients for an RF-based UAV classification has a lower time cost overscattering coefficients. Also, exploiting scalogram has a higher time cost over scattergram.
OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 28 of 32avelet transform analytics for UAV detection
Table 8
Computational time cost for classifier’s inference given in seconds.s/n Algorithm Continuous Wavelet Transform Wavelet Scattering Transform3 Classes 10 Classes 3 Classes 10 ClassesTransient Steady Transient Steady Transient Steady Transient Steady1 k NN 0.108 0.08 0.0873 0.075 0.822 0.771 0.726 0.812 SVM 0.0553 0.0243 0.109 0.083 0.155 0.117 0.423 0.4053 Ensemble 0.0568 0.357 0.379 0.094 0.129 0.101 1.48 0.1084 k NN + PCA 0.061 0.107 0.0277 0.221 0.168 0.097 0.188 0.0975 SVM + PCA 0.081 0.0263 0.11 0.061 0.167 0.075 1.027 0.0826 Ensemble + PCA 0.187 0.7324 0.046 0.063 0.778 0.124 1.18 0.2897 SqueezeNet 0.2 0.18 0.192 0.19 0.15 0.151 0.16 0.159
8. Conclusion
In this paper, we introduced an approach for the detection and identification of UAVs by exploiting theRF signals transmitted by the UAV controller. The communication between a UAV and its flight controlleroperates at the 2.4 GHz frequency which is the same frequency that WiFi and Bluetooth devices operate. Weacquired RF signals from six UAV controllers, two WiFi routers, and two Bluetooth devices. We extractedRF fingerprints from these signals by proposing four methods for extracting a signature from the two-statesof the RF signals (transient and steady state of the signal) using wavelet transform analytics (CWT and WST)to differentiate UAV signals from WiFi and Bluetooth signals, and even identify UAV types.From CWT, two approaches are proposed for feature extraction (wavelet coefficient based and the otheris image-based (i.e., scalogram)). Likewise, in WST, we come up with two methods for feature extraction(scattering coefficient based and image-based (scattergram)). Comparing each method, we realized that us-ing an image-based signature to train a pre-trained CNN based model (SqueezeNet) outperforms coefficientbased classifiers where classical ML algorithms are used. We obtained an accuracy of . at 10 dB usingthe scattergram of RF signals.In the results, we observed that using wavelet or scattering coefficient for feature extraction is sensitiveto variation in SNR because some classifiers failed outrightly when the test signals are at an SNR differentfrom the training signals despite having high accuracy at 30 dB.Finally, we showed that both transient and steady state of an RF signal carries unique attributes that canbe exploited for UAV identification or classification. It is observed that using wavelet transform analyticsfor the extraction of RF fingerprints on the steady state of RF signals can tolerate varying SNR compared tousing transient state when we evaluated the performance of our classifiers under AWGN condition. OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 29 of 32avelet transform analytics for UAV detection
In the future, we will be exploring the possibility of using data mining approaches rather than usingsignal processing approaches such as wavelet analytics for extracting signatures for UAV identification. Wewant to have a classifier that is resilient to a low level of SNR.
References [1] B. H. Y. Alsalam, K. Morton, D. Campbell, F. Gonzalez, Autonomous UAV with vision based on-board decision making for remote sensingand precision agriculture, in: Proc. IEEE Aerosp. Conf., Big Sky, MT, USA, 2017, pp. 1–12. doi:10.1109/AERO.2017.7943593 .[2] A. Banaszek, A. Zarnowski, A. Cellmer, S. Banaszek, Application of new technology data acquisition using aerial UAV digital images for theneeds of urban revitalization, in: Proc. Environmental Engineering, Int. Conf., Vilnius, Lithuania, 2017, pp. 1–7. doi:10.3846/enviro.2017.159 .[3] S. Coveney, K. Roberts, Lightweight UAV digital elevation models and orthoimagery for environmental applications: data accuracy evaluationand potential for river flood risk modelling, Int. Journal of Remote Sensing 38 (8-10) (2017) 3159–3180.[4] FAA, UAS by the numbers, "Accessed: 2020-10-12" (2020).URL [5] M. S. Schmidt, M. D. Shear, A drone, too small for radar to detect, rattles the white house, accessed: 2020-10-12 (2016).URL shorturl.at/eENOZ [6] G. Harkins, Illicit drone flights surge along U.S.-Mexico border as smugglers hunt for soft spots, accessed: 2020-10-12.URL shorturl.at/moqG1 [7] FAA, UAS sightings report, accessed: 2020-10-12 (2020).URL [8] G. C. Birch, J. C. Griffin, M. K. Erdman, UAS detection, classification, and neutralization: Market survey 2015, Sandia National Laboratories(2015).[9] D. Doroftei, G. De Cubber, Qualitative and quantitative validation of drone detection systems, in: Proc. Int. Symposium on Measurement andControl in Robotics, no. 21st, Mons, Belgium, 2018, pp. 26–28.[10] R. L. Sturdivant, E. K. Chong, Systems engineering baseline concept of a multispectral drone detection solution for airports, IEEE Access 5(2017) 7123–7138. doi:10.1109/ACCESS.2017.2697979 .[11] B. Brown, K. Buckler, Pondering personal privacy: a pragmatic approach to the fourth amendment protection of privacy in the informationage, Contemporary Justice Review 20 (2) (2017) 227–254.[12] P. Nguyen, H. Truong, M. Ravindranathan, A. Nguyen, R. Han, T. Vu, Cost-effective and passive RF-based drone presence detection andcharacterization, GetMobile: Mobile Computing and Commun. 21 (4) (2018) 30–34.[13] J. Mezei, V. Fiaska, A. Molnár, Drone sound detection, in: Proc. IEEE Int. Symposium on Computational Intelligence and Inf. (CINTI),Budapest, Hungary, 2015, pp. 333–338.[14] J. Mezei, A. Molnár, Drone sound detection by correlation, in: Proc. IEEE Int. Symposium on Applied Computational Intelligence and Inf.(SACI), Timisoara, Romania, 2016, pp. 509–518. doi:10.1109/SACI.2016.7507430 .[15] M. Nijim, N. Mantrawadi, Drone classification and identification system by phenome analysis using data mining techniques, in: Proc. IEEESymposium on Technologies for Homeland Security (HST), Waltham, MA, USA, 2016, pp. 1–5. doi:10.1109/THS.2016.7568949 .[16] X. Yue, Y. Liu, J. Wang, H. Song, H. Cao, Software defined radio and wireless acoustic networking for amateur drone surveillance, IEEECommun. Mag. 56 (4) (2018) 90–97.
OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 30 of 32avelet transform analytics for UAV detection [17] X. Shi, C. Yang, W. Xie, C. Liang, Z. Shi, J. Chen, Anti-drone system with multiple surveillance technologies: Architecture, implementation,and challenges, IEEE Commun. Mag. 56 (4) (2018) 68–74.[18] M. Saqib, S. D. Khan, N. Sharma, M. Blumenstein, A study on detecting drones using deep convolutional neural networks, in: Proc. IEEEInt. Conf. on Adv. Video and Signal Based Surveillance (AVSS), Lecce, Italy, 2017, pp. 1–6. doi:10.1109/AVSS.2017.8078558 .[19] A. Schumann, L. Sommer, J. Klatte, T. Schuchert, J. Beyerer, Deep cross-domain flying object classification for robust UAV detection, in:Proc. IEEE Int. Conf. on Advanced Video and Signal Based Surveillance (AVSS), Lecce, Italy, 2017, pp. 1–6. doi:10.1109/AVSS.2017.8078558 .[20] E. Unlu, E. Zenou, N. Riviere, P.-E. Dupouy, Deep learning-based strategies for the detection and tracking of drones using several cameras,IPSJ Trans. on Comput. Vision and Applications 11 (1) (2019) 7.[21] P. Andraši, T. Radišić, M. Muštra, J. Ivošević, Night-time detection of UAVs using thermal infrared camera, Transportation Research Procedia28 (2017) 183–190.[22] J. Drozdowicz, M. Wielgo, P. Samczynski, K. Kulpa, J. Krzonkalla, M. Mordzonek, M. Bryl, Z. Jakielaszek, 35 GHz FMCW drone detectionsystem, in: Proc. Int. Radar Symposium (IRS), Krakow, Poland, 2016, pp. 1–4. doi:10.1109/IRS.2016.7497351 .[23] G. J. Mendis, T. Randeny, J. Wei, A. Madanayake, Deep learning based doppler radar for micro UAS detection and classification, in: Proc.IEEE Military Commun. Conf., Baltimore, MD, USA, 2016, pp. 924–929. doi:10.1109/MILCOM.2016.7795448 .[24] C. Zhao, C. Chen, Z. Cai, M. Shi, X. Du, M. Guizani, Classification of small UAVs based on auxiliary classifier wasserstein GANs, in:Proc. IEEE Global Commun. Conf. (GLOBECOM), Abu Dhabi, United Arab Emirates, 2018, pp. 206–212. doi:10.1109/GLOCOM.2018.8647973 .[25] W. Zhou, L. Wang, B. Lu, N. Jin, L. Guo, J. Liu, H. Sun, H. Liu, Unmanned aerial vehicle detection based on channel state information, in:IEEE Int. Conf. on Sensing, Commun. Networking (SECON Workshops), Hong Kong, China, 2018, pp. 1–5. doi:10.1109/SECONW.2018.8396360 .[26] M. Ezuma, F. Erden, C. K. Anjinappa, O. Ozdemir, I. Guvenc, Micro-UAV detection and classification from RF fingerprints using machinelearning techniques, in: Proc. IEEE Aerosp. Conf., Big Sky, Montana, 2019.[27] M. Ezuma, F. Erden, C. K. Anjinappa, O. Ozdemir, I. Guvenc, Detection and classification of UAVs using RF fingerprints in the presence ofWiFi and bluetooth interference, IEEE Open Journal of the Commun. Society 1 (2019) 60–76. doi:10.1109/OJCOMS.2019.2955889 .[28] M. F. Al-Sa’d, A. Al-Ali, A. Mohamed, T. Khattab, A. Erbad, RF-based drone detection and identification using deep learning approaches:An initiative towards a large open source drone database, Future Generation Comput. Syst. 100 (2019) 86–97.[29] A. Alipour-Fanid, M. Dabaghchian, N. Wang, P. Wang, L. Zhao, K. Zeng, Machine learning-based delay-aware UAV detection and operationmode identification over encrypted WiFi traffic, IEEE Trans. on Inf. Forensics and Security 15 (2019) 2346–2360.[30] I. O. Kennedy, P. Scanlon, F. J. Mullany, M. M. Buddhikot, K. E. Nolan, T. W. Rondeau, Radio transmitter fingerprinting: A steady statefrequency domain approach, in: Proc. IEEE Veh. Technol. Conf., Calgary, BC, Canada, 2008, pp. 1–5. doi:10.1109/VETECF.2008.291 .[31] R. W. Klein, M. A. Temple, M. J. Mendenhall, Application of wavelet-based RF fingerprinting to enhance wireless network security, Journalof Commun. and Net. 11 (6) (2009) 544–555.[32] B. D. Fulcher, N. S. Jones, Highly comparative feature-based time-series classification, IEEE Trans. on Knowledge and Data Engineering26 (12) (2014) 3026–3037.[33] X. Wang, A. Mueen, H. Ding, G. Trajcevski, P. Scheuermann, E. Keogh, Experimental comparison of representation methods and distancemeasures for time series data, Data Mining and Knowledge Discovery 26 (2) (2013) 275–309.[34] J. Hall, M. Barbeau, E. Kranakis, Detection of transient in radio frequency fingerprinting using signal phase, Wireless and Optical Commun.(2003) 13–18.
OO Medaiyese et al.:
Preprint submitted to Elsevier
Page 31 of 32avelet transform analytics for UAV detection [35] M. S. Allahham, T. Khattab, A. Mohamed, Deep learning for RF-based drone detection and identification: A multi-channel 1-d convolutionalneural networks approach, in: Proc. IEEE Int. Conf. on Inf., IoT, and Enabling Technologies (ICIoT), Doha, Qatar, 2020, pp. 112–117. doi:10.1109/ICIoT48696.2020.9089657 .[36] E. Ozturk, F. Erden, I. Guvenc, RF-based low-SNR classification of UAVs using convolutional neural networks, arXiv preprintarXiv:2009.05519 (2020).[37] S. Mallat, A wavelet tour of signal processing, Elsevier, 1999.[38] P. S. Adisson, The illustrated wavelet transform handbook: Introductory theory and applications in science, engineering, medicine and finance(2002).[39] O. Rioul, M. Vetterli, Wavelets and signal processing, IEEE Signal Processing Mag. 8 (4) (1991) 14–38.[40] R. X. Gao, R. Yan, Wavelets: Theory and applications for manufacturing, Springer Science & Business Media, 2010.[41] J. Andén, S. Mallat, Deep scattering spectrum, IEEE Trans.on Signal Processing 62 (16) (2014) 4114–4128.[42] J. Bruna, S. Mallat, Invariant scattering convolution networks, IEEE Trans. on Pattern Analysis and Machine Intelligence 35 (8) (2013)1872–1886.[43] S. Mallat, Group invariant scattering, Commun. on Pure and Applied Mathematics 65 (10) (2012) 1331–1398.[44] M. Stéphane, Understanding deep convolutional networks, Philosophical Transactions of the Royal Society A: Mathematical, Physical andEngineering Sciences 374 (2065) (2016) 20150203.[45] L. Shen, L. Bai, A review on gabor wavelets for face recognition, Pattern analysis and applications 9 (2) (2006) 273–292.[46] F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, K. Keutzer, SqueezeNet: AlexNet-level accuracy with 50x fewer parametersand< 0.5 MB model size, arXiv preprint arXiv:1602.07360 (2016).[47] S. J. Pan, Q. Yang, A survey on transfer learning, IEEE Trans. on Knowledge and Data Engineering 22 (10) (2009) 1345–1359.[48] J. Snoek, H. Larochelle, R. P. Adams, Practical bayesian optimization of machine learning algorithms, Advances in Neural Inf. Processingsyst. 25 (2012) 2951–2959.[49] M. Kantardzic, Data mining: concepts, models, methods, and algorithms, John Wiley & Sons, 2011.
OO Medaiyese et al.: