Huabiao Lu

ENDMal: An anti-obfuscation and collaborative malware detection system using syscall sequences

Abstract Malware obfuscation obscures malware into different versions, making traditional syntactic nature based detection ineffective. Furthermore, with the huge and exponentially growing number of malware samples, existing malware detection systems are either evaded by malware obfuscation, or overwhelmed by numerous malware samples. This paper proposes an anti-obfuscation, scalable and collaborative malware detection system—ENDMal. ENDMal identifies the program that behaves suspiciously in end-hosts and similarly between a group of suspicious programs in a wide area as malicious. We present the Iterative Sequence Alignment (ISA) method to defeat malware obfuscation. Instead of using complex behavior graph, we propose the Handle dependences and Probabilistic Ordering Dependence (HPOD) technology to represent the program behaviors. In addition, we design a novel information sharing infrastructure, RENShare, to collaboratively congregate the group characteristics of programs spreading over different network areas. Our experimental results show that ENDMal can detect unknown malwares much faster than the centralized detection system and is more effective than the existing distributed detection system.

DiffSig: resource differentiation based malware behavioral concise signature generation

Malware obfuscation obscures malware into a different form thats functionally identical to the original one, and makes syntactic signature ineffective. Furthermore, malware samples are huge and growing at an exponential pace. Behavioral signature is an effective way to defeat obfuscation. However, state-of-the-art behavioral signature, behavior graph, is although very effective but unfortunately too complicated and not scalable to handle exponential growing malware samples; in addition, it is too slow to be used as real-time detectors. This paper proposes an anti-obfuscation and scalable behavioral signature generation system, DiffSig, which voids information-flow tracking which is the chief culprit for the complex and inefficiency of graph behavior, thus, losing some data dependencies, but describes handle dependencies more accurate than graph behavior by restrict the profile type of resource that each handle dependency can reference to. Our experiment results show that DiffSig is scalable and efficient, and can detect new malware samples effectively.

Generating Lightweight Behavioral Signature for Malware Detection in People-Centric Sensing

People-centric sensing (PCS) is an emerging paradigm of sensor network which turns daily used mobile devices (such as smartphones and PDAs) to sensors. It is promising but faces severe security problems. As smartphones are already and will keep up to be attractive targets to attackers, even more, with strong connectivity and homogeneous applications, all mobile devices in PCS will risk being infected by malware more rapidly. Even worse, attackers usually obfuscate their malwares in order to avoid simple (syntactic signature based) detection. Thus, more intelligent (behavioral signature based) detection is needed. But in the field of network security, the state-of-the-art behavioral signature—behavior graph—is too complicated to be used in mobile devices. This paper proposes a novel behavioral signature generation system—SimBehavior—to generate lightweight behavioral signature for malware detection in PCS. Generated lightweight behavioral signature is a bit like regex (regular expression) rules. And thus, unlike malware detection using behavior graph is NP-Complete, using our lightweight behavioral signature is efficient and very suitable for malware detection in PCS. Our experimental results show that SimBehavior can extract behavioral signatures effectively, and generated lightweight behavioral signatures can be used to detect new malware samples in PCS efficiently and effectively.

Mining Network Behavior Specifications of Malware Based on Binary Analysis

Nowadays, malware, especially for a botnet, heavily employs network communication to accomplish predefined malicious functionalities. The network behavior of malware attracts attention of researchers. However, the network traffic used for network-based signatures generation and botnet detection is captured passively from an execution environment, that there are several limitations. In this paper, we present a network behavior mining approach based on binary analysis, named NBSBA. Our goal is to accurately understand the network behavior of malware in details, capture the packets the malware sample under analysis launched as soon as possible, and extract network behavior of malware as completely as possible. We firstly give a network behavior specification and then describe the NBSBA. And we implement a prototype system to evaluate the NBSBA. The experiment demonstrates that our approach is efficient.