Matias Madou

Deobfuscation: reverse engineering obfuscated code

In recent years, code obfuscation has attracted attention as a low cost approach to improving software security by making it difficult for attackers to understand the inner workings of proprietary software systems. This paper examines techniques for automatic deobfuscation of obfuscated programs, as a step towards reverse engineering such programs. Our results indicate that much of the effects of code obfuscation, designed to increase the difficulty of static analyses, can be defeated using simple combinations of straightforward static and dynamic analyses. Our results have applications to both software engineering and software security. In the context of software engineering, we show how dynamic analyses can be used to enhance reverse engineering, even for code that has been designed to be difficult to reverse engineer. For software security, our results serve as an attack model for code obfuscators, and can help with the development of obfuscation techniques that are more resilient to straightforward reverse engineering.

Software protection through dynamic code mutation

Reverse engineering of executable programs, by disassembling them and then using program analyses to recover high level semantic information, plays an important role in attacks against software systems, and can facilitate software piracy. This paper introduces a novel technique to complicate reverse engineering. The idea is to change the program code repeatedly as it executes, thereby thwarting correct disassembly. The technique can be made as secure as the least secure component of opaque variables and pseudorandom number generators.

Program obfuscation: a quantitative approach

Despite the recent advances in the theory underlying obfuscation, there still is a need to evaluate the quality of practical obfuscating transformations more quickly and easily. This paper presents the first steps toward a comprehensive evaluation suite consisting of a number of deobfuscating transformations and complexity metrics that can be readily applied on existing and future transformations in the domain of binary obfuscation. In particular, a framework based on software complexity metrics measuring four program properties: code, control flow, data and data flow is suggested. A number of well-known obfuscating and deobfuscating transformations are evaluated based upon their impact on a set of complexity metrics. This enables us to quantitatively evaluate the potency of the (de)obfuscating transformations.

Hybrid static-dynamic attacks against software protection mechanisms

Advances in reverse engineering and program analyses have made software extremely vulnerable to malicious host attacks. These attacks typically take the form of intellectual property violations, against which the software needs to be protected. The intellectual property that needs to be protected can take on different forms. The software might, e.g., consist itself of proprietary algorithms and datastructures or it could provide controlled access to copyrighted material. Therefore, in recent years, a number of techniques have been explored to protect software. Many of these techniques provide a reasonable level of security against static-only attacks. Many of them however fail to address the problem of dynamic or hybrid static-dynamic attacks. While this type of attack is already commonly used by black-hats, this is one of the first scientific papers to discuss the potential of these attacks through which an attacker can analyze, control and modify a program extensively. The concepts are illustrated through a case study of a recently proposed algorithm for software watermarking [6].

Opaque predicates detection by abstract interpretation

Code obfuscation and software watermarking are well known techniques designed to prevent the illegal reuse of software. Code obfuscation prevents malicious reverse engineering, while software watermarking protects code from piracy. An interesting class of algorithms for code obfuscation and software watermarking relies on the insertion of opaque predicates. It turns out that attackers based on a dynamic or an hybrid static-dynamic approach are either not precise or time consuming in eliminating opaque predicates. We present an abstract interpretation-based methodology for removing opaque predicates from programs. Abstract interpretation provides the right framework for proving the correctness of our approach, together with a general methodology for designing efficient attackers for a relevant class of opaque predicates. Experimental evaluations show that abstract interpretation based attacks significantly reduce the time needed to eliminate opaque predicates.

LOCO: an interactive code (De)obfuscation tool

This paper presents LOCO, a graphical, interactive environment to experiment with code obfuscation and deobfuscation transformations, which can be applied automatically, semi-automatically and by hand. LOCO is an extension of the multi-platform visualization tool LANCET, combined with an obfuscation infrastructure in the underlying link-time program rewriter DIABLO. By use of LOCO, a developer can easily navigate through the control flow graph of a program and do fine-grained obfuscation, test new obfuscation transformations, test the robustness of existing transformations or improve existing transformations.

Towards tamper resistant code encryption: practice and experience

In recent years, many have suggested to apply encryption in the domain of software protection against malicious hosts. However, little information seems to be available on the implementation aspects or cost of the different schemes. This paper tries to fill the gap by presenting our experience with several encryption techniques: bulk encryption, an on-demand decryption scheme, and a combination of both techniques. Our scheme offers maximal protection against both static and dynamic code analysis and tampering. We validate our techniques by applying them on several benchmark programs of the CPU2006 Test Suite. And finally, we propose a heuristic which trades off security versus performance, resulting in a decrease of the runtime overhead.

Matching Control Flow of Program Versions

In many application areas, including piracy detection, software debugging and maintenance, situations arise in which there is a need for comparing two versions of a program that dynamically behave the same even though they statically appear to be different. Recently dynamic matching [18] was proposed by us which uses execution histories to automatically produce mappings between instructions in the two program versions. The mappings then can be used to understand the correspondence between the two versions by a user involved in software piracy detection or a comparison checker involved in debugging of optimized code. However, if a programs control flow is substantially altered, which usually occurs in obfuscation or even manual transformations, mappings at instruction level are not sufficient to enable a good understanding of the correspondence. In this paper, we present a comprehensive dynamic matching algorithm with the focus on call graph and control flow matching. Our technique works in the presence of aggressive control flow transformations (both interprocedural such as function Mining/outlining and intraprocedural such as control flow flattening) and produces mappings of interprocedural and intraprocedural control flow in addition to mapping between instructions. We evaluated our dynamic matching algorithms by attempting to match original program with versions that were subjected to popular obfuscation and control flow altering transformations. Our experimental results show that the control flow mappings produced are highly accurate and complete, for the programs considered.

A model for self-modifying code

Self-modifying code is notoriously hard to understand and therefore very well suited to hide program internals. In this paper we introduce a program representation for this type of code: the state-enhanced control flow graph. It is shown how this program representation can be constructed, how it can be linearized into a binary program, and how it can be used to generate, analyze and transform self-modifying code.

Understanding Obfuscated Code

Code obfuscation makes it harder for a security analyst to understand the malicious payload of a program. In most cases an analyst needs to study the program at the machine code level, with little or no extra information available, apart from his experience. An unexperienced analyst is confronted with a steep learning curve, as understanding unobfuscated machine code already requires some skills. We have built Loco, a graphical, interactive environment to help a security analyst improving his skills in understanding obfuscated code