Davide Maiorca

Evasion attacks against machine learning at test time

In security-sensitive applications, the success of machine learning depends on a thorough vetting of their resistance to adversarial data. In one pertinent, well-motivated attack scenario, an adversary may attempt to evade a deployed system at test time by carefully manipulating attack samples. In this work, we present a simple but effective gradient-based approach that can be exploited to systematically assess the security of several, widely-used classification algorithms against evasion attacks. Following a recently proposed framework for security evaluation, we simulate attack scenarios that exhibit different risk levels for the classifier by increasing the attackers knowledge of the system and her ability to manipulate attack samples. This gives the classifier designer a better picture of the classifier performance under evasion attacks, and allows him to perform a more informed model selection (or parameter setting). We evaluate our approach on the relevant security task of malware detection in PDF files, and show that such systems can be easily evaded. We also sketch some countermeasures suggested by our analysis.

Looking at the bag is not enough to find the bomb: an evasion of structural methods for malicious PDF files detection

PDF files have proved to be excellent malicious-code bearing vectors. Thanks to their flexible logical structure, an attack can be hidden in several ways, and easily deceive protection mechanisms based on file-type filtering. Recent work showed that malicious PDF files can be accurately detected by analyzing their logical structure, with excellent results. In this paper, we present and practically demonstrate a novel evasion technique, called reverse mimicry, that can easily defeat such kind of analysis. We implement it using real samples and validate our approach by testing it against various PDF malware detectors proposed so far. Finally, we highlight the importance of developing systems robust to adversarial attacks and propose a framework to strengthen PDF malware detection against evasion.

A pattern recognition system for malicious PDF files detection

Malicious PDF files have been used to harm computer security during the past two-three years, and modern antivirus are proving to be not completely effective against this kind of threat. In this paper an innovative technique, which combines a feature extractor module strongly related to the structure of PDF files and an effective classifier, is presented. This system has proven to be more effective than other state-of-the-art research tools for malicious PDF detection, as well as than most of antivirus in commerce. Moreover, its flexibility allows adopting it either as a stand-alone tool or as plug-in to improve the performance of an already installed antivirus.

Security Evaluation of Support Vector Machines in Adversarial Environments

Support vector machines (SVMs) are among the most popular classification techniques adopted in security applications like malware detection, intrusion detection, and spam filtering. However, if SVMs are to be incorporated in real-world security systems, they must be able to cope with attack patterns that can either mislead the learning algorithm (poisoning), evade detection (evasion) or gain information about their internal parameters (privacy breaches). The main contributions of this chapter are twofold. First, we introduce a formal general framework for the empirical evaluation of the security of machine-learning systems. Second, according to our framework, we demonstrate the feasibility of evasion, poisoning and privacy attacks against SVMs in real-world security problems. For each attack technique, we evaluate its impact and discuss whether (and how) it can be countered through an adversary-aware design of SVMs. Our experiments are easily reproducible thanks to open-source code that we have made available, together with all the employed datasets, on a public repository.

Yes, Machine Learning Can Be More Secure! A Case Study on Android Malware Detection

To cope with the increasing variability and sophistication of modern attacks, machine learning has been widely adopted as a statistically-sound tool for malware detection. However, its security against well-crafted attacks has not only been recently questioned, but it has been shown that machine learning exhibits inherent vulnerabilities that can be exploited to evade detection at test time. In other words, machine learning itself can be the weakest link in a security system. In this paper, we rely upon a previously-proposed attack framework to categorize potential attack scenarios against learning-based malware detection tools, by modeling attackers with different skills and capabilities. We then define and implement a set of corresponding evasion attacks to thoroughly assess the security of Drebin, an Android malware detector. The main contribution of this work is the proposal of a simple and scalable secure-learning paradigm that mitigates the impact of evasion attacks, while only slightly worsening the detection rate in the absence of attack. We finally argue that our secure-learning approach can also be readily applied to other malware detection tasks.

Stealth attacks

In order to effectively evade anti-malware solutions, Android malware authors are progressively resorting to automatic obfuscation strategies. Recent works have shown, on small-scale experiments, the possibility of evading anti-malware engines by applying simple obfuscation transformations on previously detected malware samples. In this paper, we provide a large-scale experiment in which the detection performances of a high number of anti-malware solutions are tested against two different sets of malware samples that have been obfuscated according to different strategies. Moreover, we show that anti-malware engines search for possible malicious content inside assets and entry-point classes. We also provide a temporal analysis of the detection performances of anti-malware engines to verify if their resilience has improved since 2013. Finally, we show how, by manipulating the area of the Android executable that contains the strings used by the application, it is possible to deceive anti-malware engines so that they will identify legitimate samples as malware. On one hand, the attained results show that anti-malware systems have improved their resilience against trivial obfuscation techniques. On the other hand, more complex changes to the application executable have proved to be still effective against detection. Thus, we claim that a deeper static (or dynamic) analysis of the application is needed to improve the robustness of such systems.

R-PackDroid: API package-based characterization and detection of mobile ransomware

Ransomware has become a serious and concrete threat for mobile platforms and in particular for Android. In this paper, we propose R-PackDroid, a machine learning system for the detection of Android ransomware. Differently to previous works, we leverage information extracted from system API packages, which allow to characterize applications without specific knowledge of user-defined content such as the application language or strings. Results attained on very recent data show that it is possible to detect Android ransomware and to distinguish it from generic malware with very high accuracy. Moreover, we used R-PackDroid to flag applications that were detected as ransomware with very low confidence by the VirusTotal service. In this way, we were able to correctly distinguish true ransomware from false positives, thus providing valuable help for the analysis of these malicious applications.

A structural and content-based approach for a precise and robust detection of malicious PDF files

During the past years, malicious PDF files have become a serious threat for the security of modern computer systems. They are characterized by a complex structure and their variety is considerably high. Several solutions have been academically developed to mitigate such attacks. However, they leveraged on information that were extracted from either only the structure or the content of the PDF file. This creates problems when trying to detect non-Javascript or targeted attacks. In this paper, we present a novel machine learning system for the automatic detection of malicious PDF documents. It extracts information from both the structure and the content of the PDF file, and it features an advanced parsing mechanism. In this way, it is possible to detect a wide variety of attacks, including non-Javascript and parsing-based ones. Moreover, with a careful choice of the learning algorithm, our approach provides a significantly higher accuracy compared to other static analysis techniques, especially in the presence of adversarial malware manipulation.

Evaluating Analysis Tools for Android Apps: Status Quo and Robustness Against Obfuscation

The recent past has shown that Android smartphones became the most popular target for malware authors. Malware families offer a variety of features that allow, among the others, to steal arbitrary data and to cause significant monetary losses. This circumstances led to the development of many different analysis methods that are aimed to assess the absence of potential harm or malicious behavior in mobile apps. In return, malware authors devised more sophisticated methods to write mobile malware that attempt to thwart such analyses. In this work, we briefly describe assumptions analysis tools rely on to detect malicious content and behavior. We then present results of a new obfuscation framework that aims to break such assumptions, thus modifying Android apps to avoid them being analyzed by the targeted systems. We use our framework to evaluate the robustness of static and dynamic analysis systems for Android apps against such transformations.

An Evasion Resilient Approach to the Detection of Malicious PDF Files

Malicious PDF files still constitute a serious threat to the systems security. New reader vulnerabilities have been discovered, and research has shown that current state of the art approaches can be easily bypassed by exploiting weaknesses caused by erroneous parsing or incomplete information extraction. In this work, we present a novel machine learning system to the detection of malicious PDF files. We have developed a static approach that leverages on information extracted by both the structure and the content of PDF files, which allows to improve the system robustness against evasion attacks. Experimental results show that our system is able to outperform all publicly available state of the art tools. We also report a significant improvement of the performances at detecting reverse mimicry attacks, which are able to completely evade systems that only extract information from the PDF file structure. Finally, we claim that, to avoid targeted attacks, a more careful design of machine learning based detectors is needed.