Andrey M. Dolgikh

Expressive, efficient and obfuscation resilient behavior based IDS

Behavior based intrusion detection systems (BIDS) offer the only effective solution against modern malware. While dynamic BIDS have obvious advantages, their success hinges upon three interrelated factors: signature expressiveness, vulnerability to behavioral obfuscation and run-time efficiency of signature matching. To achieve higher signature expressiveness, a new approach for formal specification of the malicious functionalities based on abstract activity diagrams (AD) which incorporate multiple realizations of the specified functionality. We analyzed both inter and intra-process behavioral obfuscation techniques that can compromise existing BIDS. As a solution, we proposed specification generalization that implies augmenting (generalizing) otherwise obfuscation prone specification into more generic, obfuscation resilient specification. We suggest colored Petri nets as a basis for functionality recognition at the system call level. We implemented a prototype IDS that has been evaluated on malicious and legitimate programs. The experimental results indicated extremely low false positives and negatives. Moreover, the IDS shows very low execution overhead and negligible overhead penalty due to anti-obfuscation generalization.

Unmanned Aerial Vehicle security using Recursive parameter estimation

The proliferation of Unmanned Aerial Vehicles (UAVs) raises a host of new security concerns. Our research resulted in a prototype UAV monitoring system, which captures flight data and performs real-time estimation/tracking of airframe and controller parameters utilizing the Recursive Least Squares Method. Subjected to statistical validation and trend analysis, parameter estimates are instrumental for the detection of some classes of cyber attacks and incipient hardware failures that can invariably jeopardize mission success. Our results demonstrate that achieving efficient anomaly detection during flight is possible through the intelligent application of statistical methods to system behavioral profiling.

Unmanned Aerial Vehicle security using behavioral profiling

The proliferation of Unmanned Aerial Vehicles (UAVs) raises a host of new security concerns. Our research resulted in a prototype UAV monitoring system, which captures flight data and performs real-time behavioral monitoring. If a behavioral anomaly is found, the system will alert the operator. In addition, a novel visualization system tracks the real time behavior and flight of multiple UAVs. Our results demonstrate that application of behavioral methods facilitates efficient anomaly detection during flight, thus providing additional assurance of the mission success.

Cloud Security Auditing Based on Behavioral Modeling

Multi-tenancy is one of the most attractive features of cloud computing, which provides significant benefits to both clients and service providers by supporting elastic, efficient, and on-demand resource provisioning and allocation. However, this architecture also introduces additional security implications. Client Virtual Machine (VM) instances running on the same physical machine are susceptible to side-channel and escape-to-hypervisor attacks. The timely prevention of intrusive behavior and malicious processes using signature based intrusion detection technologies, or system call level anomaly analysis is a very challenging task due to a high rate of false alarms. In this work, a behavioral modeling scheme is proposed to audit the behaviors of client VMs and to detect suspicious processes on the highest semantic level. Our preliminary results have validated the effectiveness and efficiency of this novel approach.

Using behavioral modeling and customized normalcy profiles as protection against targeted cyber-attacks

Targeted cyber-attacks present significant threat to modern computing systems. Modern industrial control systems (SCADA) or military networks are example of high value targets with potentially severe implications in case of successful attack. Anomaly detection can provide solution to targeted attacks as attack is likely to introduce some distortion to observable system activity. Most of the anomaly detection has been done on the level of sequences of system calls and is known to have problems with high false alarm rates. In this paper, we show that better results can be obtained by performing behavioral analysis on higher semantic level. We observe that many critical computer systems serve a specific purpose and are expected to run strictly limited sets of software. We model this behavior by creating customized normalcy profile of this system and evaluate how well does anomaly based detection work in this scenario.

Dynamic, resilient detection of complex malicious functionalities in the system call domain

A novel approach to malware detection by recognizing known inter-process and intra-process malicious functionalities in software behavior is proposed. It encompasses two essential tasks: the specification of a functionality that may involve a joint activity of several apparently independent processes, and efficient recognition of the specified functionality in the process behavior. The robustness of the proposed technology is achieved by the generalization of the specification domain that is separated from the detection domain. The functionalities of interest are defined in the abstract system domain through activity diagrams, thus resulting in formal specifications that are rather generic and less prone to false negatives. To facilitate the detection, we developed a procedure that automatically generates a Colored Petri Net recognizing the specified functionality in the system call domain. The separation of specification and recognition domains results in signature expressiveness and recognition efficiency. The approach is illustrated by the analysis, specification and consequent recognition of several common malicious functionalities including self-replication engines and popular payloads. A prototype IDS implementing the proposed approach has been developed and successfully tested on a set of real malware.

Detection of Worm Propagation Engines in the System Call Domain using Colored Petri Nets

While network worms carry various payloads and may utilize any available exploits, they all have one common component - the propagation engine. Moreover, it is important to note that the number of conceptually distinct propagation engines employed by existing network worms is quite limited. This paper presents a novel signature-based approach for detecting attacks perpetrated by network worms as a manifestation of a semantic functionality performed by one of the few known propagation engines. We propose a novel methodology to recognize any semantic functionality in the system call domain through utilizing colored Petri Nets. In this application, Petri Nets embody behavior-based signatures of the propagation engine functionalities. These signatures are indicative of the shell code activity in the first stage of the worm proliferation. We developed, tested and evaluated a propagation engine detector (PED) system that detects activity of the worm shell code executed by a process during an attack. Moreover, PED is able to recognize the type of propagation engine employed by the attacking worm.

Behavioral Modeling for Suspicious Process Detection in Cloud Computing Environments

One of the defining features of cloud computing, multi-tenancy provides significant benefits to both clients and service providers by supporting elastic on-demand resource provisioning and efficient resource allocation. However, this architecture also introduces additional security implications. Client virtual machine (VM) instances running on the same physical machine are susceptible to side-channel and escape-to-hypervisor attacks. Timely detection/mitigation of intrusive behaviors of malicious processes using signature based intrusion detection technologies or system call level anomaly analysis due to high false alarm rate presents a challenging task. In this work, a behavioral modeling scheme is proposed to detect suspicious processes on the highest semantic level. Our preliminary results have validated the effectiveness and efficiency of this novel approach.

Cloud security auditing based on behavioural modelling

Multi-tenancy is one of the most attractive features of cloud computing, which provides significant benefits to both clients and service providers by supporting elastic, efficient and on-demand resource provisioning and allocation. Multi-tenancy also introduces additional security auditing opportunities. Security auditing can be consolidated and offloaded onto a dedicated and well-protected service. The timely prevention of intrusive behaviour and malicious processes using signature-based intrusion detection technologies or system call level anomaly analysis is a very challenging task due to a high rate of false alarms. In this work, a behavioural modelling scheme is proposed to audit the behaviours of client virtual machine and detect suspicious processes on the level of functionality. The proposed scheme can be used as a community security auditing service. The scheme can also be used by cloud providers to offer automatic identification and auditing of the tenant’s services. Our preliminary results have...

Customized normalcy profiles for the detection of targeted attacks

Functionality is the highest semantic level of the software behavior pyramid that reflects goals of the software rather than its specific implementation. Detection of malicious functionalities presents an effective way to detect malware in behavior-based IDS. A technology for mining system call data, discussed herein, results in the detection of functionalities representing operation of legitimate software within a closed network environment. The set of such functionalities combined with the frequencies of their execution constitutes a normalcy profile typical for this environment. Detection of deviations from this normalcy profile, new functionalities and/or changes in the execution frequencies, provides evidence of abnormal activity in the network caused by malware. This approach could be especially valuable for the detection of targeted zero-day attacks. The paper presents the results of the implementation and testing of the described technology on the computer network testbed.