Fabio Di Troia

A comparison of static, dynamic, and hybrid analysis for malware detection

In this research, we compare malware detection techniques based on static, dynamic, and hybrid analysis. Specifically, we train Hidden Markov Models (HMMs) on both static and dynamic feature sets and compare the resulting detection rates over a substantial number of malware families. We also consider hybrid cases, where dynamic analysis is used in the training phase, with static techniques used in the detection phase, and vice versa. In our experiments, a fully dynamic approach generally yields the best detection rates. We discuss the implications of this research for malware detection based on hybrid techniques.

Support vector machines and malware detection

In this research, we test three advanced malware scoring techniques that have shown promise in previous research, namely, Hidden Markov Models, Simple Substitution Distance, and Opcode Graph based detection. We then perform a careful robustness analysis by employing morphing strategies that cause each score to fail. We show that combining scores using a Support Vector Machine yields results that are significantly more robust than those obtained using any of the individual scores.

Clustering for malware classification

In this research, we apply clustering techniques to the malware classification problem. We compute clusters using the well-known K-means and Expectation Maximization algorithms, with the underlying scores based on Hidden Markov Models. We compare the results obtained from these two clustering approaches and we carefully consider the interplay between the dimension (i.e., number of models used for clustering), and the number of clusters, with respect to the accuracy of the clustering.

Malware Detection Using Dynamic Birthmarks

In this paper, we compare the effectiveness of Hidden Markov Models (HMMs) with that of Profile Hidden Markov Models (PHMMs), where both are trained on sequences of API calls. We compare our results to static analysis using HMMs trained on sequences of opcodes, and show that dynamic analysis achieves significantly stronger results in many cases. Furthermore, in comparing our two dynamic analysis approaches, we find that using PHMMs consistently outperforms our technique based on HMMs.

Classic cryptanalysis using hidden Markov models

ABSTRACT In this article, the authors present a detailed introduction to hidden Markov models (HMM). They then apply HMMs to the problem of solving simple substitution ciphers, and they empirically determine the accuracy as a function of the ciphertext length and the number of random restarts. Application to homophonic substitutions and other classic ciphers is briefly considered.

Static Analysis of Malicious Java Applets

In this research we consider the problem of detecting malicious Java applets, based on static analysis. Dynamic analysis can be more informative, since it is immune to many common obfuscation techniques, while static analysis is often more efficient, since it does not require code execution or emulation. Consequently, static analysis is generally preferred, provided the results are comparable to those obtained using dynamic analysis. We conduct experiments using three techniques that have been employed in previous studies of metamorphic malware. We show that our static approach can detect malicious Java applets with greater accuracy than previously published research that relied on dynamic analysis.

Clustering versus SVM for malware detection

Previous work has shown that cluster analysis can be used to effectively classify malware into meaningful families. In this research, we apply cluster analysis to the challenging problem of classifying previously unknown malware. We perform several experiments involving malware clustering. We compare our clustering results to those obtained when a support vector machine (SVM) is trained on the malware family. Using clustering, we are able to classify malware with an accuracy comparable to that of an SVM. An advantage of the clustering approach is that a new malware family can be classified before a model has been trained specifically for the family.

Function Call Graphs Versus Machine Learning for Malware Detection

Recent work has shown that a function call graph technique can perform well on some challenging malware detection problems. In this chapter, we compare this function call graph approach to elementary machine learning techniques that are trained on simpler features. We find that the machine learning techniques are generally more robust than the function call graphs, in the sense that the malware must be modified to a far greater extent before the machine learning techniques are significantly degraded. This work provides evidence that machine learning is likely to perform better than ad hoc approaches, particularly when faced with intelligent attackers who can attempt to exploit the inherent weaknesses in a given detection strategy.

Static and Dynamic Analysis of Android Malware

Static analysis relies on features extracted without executing code, while dynamic analysis extracts features based on execution (or emulation). In general, static analysis is more efficient, while dynamic analysis can be more informative, particularly in cases where the code is obfuscated. Static analysis of an Android application can, for example, rely on features extracted from the manifest file or the Java bytecode, while dynamic analysis of such applications might deal with features involving dynamic code loading and system calls. In this research, we apply machine learning techniques to analyze the relative effectiveness of particular static and dynamic features for detecting Android malware. We also carefully analyze the robustness of the scoring techniques under consideration.

Advanced transcriptase for JavaScript malware

Previous work has shown that JavaScript malware can manipulate its internal code with relative ease using an approach known as Transcriptase. However, the resulting malware remained susceptible to software similarity based scoring techniques. In this research, we develop and analyze an advanced version of Transcriptase that is entirely practical and is not detectable using any of several scoring techniques considered. Our technique, which is based on entropy manipulations and multiple layers of encryption, is applicable generally for use in malware obfuscation.