Robert Luh

A Survey of Visualization Systems for Malware Analysis

Due to the increasing threat from malicious software (malware), monitoring of vulnerable systems is becoming increasingly important. The need to log and analyze activity encompasses networks, individual computers, as well as mobile devices. While there are various automatic approaches and techniques available to detect, identify, or capture malware, the actual analysis of the ever-increasing number of suspicious samples is a time-consuming process for malware analysts. The use of visualization and highly interactive visual analytics systems can help to support this analysis process with respect to investigation, comparison, and summarization of malware samples. Currently, there is no survey available that reviews available visualization systems supporting this important and emerging field. We provide a systematic overview and categorization of malware visualization systems from the perspective of visual analytics. Additionally, we identify and evaluate data providers and commercial tools that produce meaningful input data for the reviewed malware visualization systems. This helps to reveal data types that are currently underrepresented, enabling new research opportunities in the visualization community.

Semantics-aware detection of targeted attacks: a survey

In today’s interconnected digital world, targeted attacks have become a serious threat to conventional computer systems and critical infrastructure alike. Many researchers contribute to the fight against network intrusions or malicious software by proposing novel detection systems or analysis methods. However, few of these solutions have a particular focus on Advanced Persistent Threats or similarly sophisticated multi-stage attacks. This turns finding domain-appropriate methodologies or developing new approaches into a major research challenge. To overcome these obstacles, we present a structured review of semantics-aware works that have a high potential for contributing to the analysis or detection of targeted attacks. We introduce a detailed literature evaluation schema in addition to a highly granular model for article categorization. Out of 123 identified papers, 60 were found to be relevant in the context of this study. The selected articles are comprehensively reviewed and assessed in accordance to Kitchenham’s guidelines for systematic literature reviews. In conclusion, we combine new insights and the status quo of current research into the concept of an ideal systemic approach capable of semantically processing and evaluating information from different observation points.

Problem characterization and abstraction for visual analytics in behavior-based malware pattern analysis

Behavior-based analysis of emerging malware families involves finding suspicious patterns in large collections of execution traces. This activity cannot be automated for previously unknown malware families and thus malware analysts would benefit greatly from integrating visual analytics methods in their process. However existing approaches are limited to fairly static representations of data and there is no systematic characterization and abstraction of this problem domain. Therefore we performed a systematic literature study, conducted a focus group as well as semi-structured interviews with 10 malware analysts to elicit a problem abstraction along the lines of data, users, and tasks. The requirements emerging from this work can serve as basis for future design proposals to visual analytics-supported malware pattern analysis.

Malicious Behavior Patterns

This paper details a schema developed for defining malicious behavior in software. The presented approach enables malware analysts to identify and categorize malicious software through its high-level goals as well as down to the individual functions executed on operating system level. We demonstrate the practical application of the schema by mapping dynamically extracted system call patterns to a comprehensive hierarchy of malicious behavior.

Classifying malicious system behavior using event propagation trees

Behavior-based analysis of dynamically executed software has become an established technique to identifying and analyzing potential malware. Most solutions rely on API or system call patterns to determine whether a sample is exhibiting malicious activity. Analysis is usually performed on demand and offers little insight into the current system state. In addition, the fixed nature of behavioral patterns is known to cause false-positives whenever a certain, potentially malicious action is used in a benign context. To combat these shortcomings, this paper proposes an analysis system capable of building event propagation trees from real-time kernel monitoring data. Distance-based anomaly detection is then used to find and highlight activities deviating from a predefined baseline established through heuristic clustering. The system was tested on a set of real-world data collected by a number of host-based agents distributed across a corporate network.

Design of an Anomaly-based Threat Detection a Explication System.

Current signature-based malware detection systems are heavily reliant on fixed patterns that struggle with unknown or evasive applications, while behavior-based solutions usually leave most of the interpretative work to a human analyst. In this paper, we propose a system able to explain anomalous behavior within a user session by considering anomalies identified through their deviation from a set of baseline process graphs. To minimize computational requirements we adapt star structures, a bipartite representation used to approximate the edit distance between two graphs. Baseline templates are generated automatically and adapt to the nature of the respective process. We prototypically implement smart anomaly explication through a number of competency questions derived and evaluated using the decision tree algorithm. The determined key factors are ultimately mapped to a dedicated APT attack stage ontology that considers actions, actors, as well as target assets.

TAON: an ontology-based approach to mitigating targeted attacks

Targeted attacks on IT systems are a rising threat against the confidentiality of sensitive data and the availability of systems and infrastructures. Planning for the eventuality of a data breach or sabotage attack has become an increasingly difficult task with the emergence of advanced persistent threats (APTs), a class of highly sophisticated cyber-attacks that are nigh impossible to detect using conventional signature-based systems. Understanding, interpreting, and correlating the particulars of such advanced targeted attacks is a major research challenge that needs to be tackled before behavior-based approaches can evolve from their current state to truly semantics-aware solutions. Ontologies offer a versatile foundation well suited for depicting the complex connections between such behavioral data and the diverse technical and organizational properties of an IT system. In order to facilitate the development of novel behavior-based detection systems, we present TAON, an OWL-based ontology offering a holistic view on actors, assets, and threat details, which are mapped to individual abstracted events and anomalies that can be detected by todays monitoring data providers. TOAN offers a straightforward means to plan an organizations defense against APTs and helps to understand how, why, and by whom certain resources are targeted. Populated by concrete data, the proposed ontology becomes a smart correlation framework able to combine several data sources into a semantic assessment of any targeted attack.

Sequitur-based Inference and Analysis Framework for Malicious System Behavior

Targeted attacks on IT systems are a rising threat against the confidentiality of sensitive data and the availability of critical systems. With the emergence of Advanced Persistent Threats (APTs), it has become more important than ever to fully understand the particulars of such attacks. Grammar inference offers a powerful foundation for the automated extraction of behavioral patterns from sequential system traces. In order to facilitate the interpretation and analysis of APTs, we present a grammar inference system based on Sequitur, a greedy compression algorithm that constructs a context-free grammar (CFG) from string-based input data. Next to recursive rule extraction, we expanded the procedure through automated assessment routines capable of dealing with multiple input sources and types. This enables the identification of relevant patterns in sequential corpora of arbitrary quantity and size. On the formal side, we extended the CFG with attributes that help depict the extracted (malicious) actions in a comprehensive fashion. The tool’s output is automatically mapped to the grammar for further parsing and discovery-focused pattern visualization.

LLR-Based Sentiment Analysis for Kernel Event Sequences

Behavior-based analysis of dynamically executed binaries has become a widely used technique for the identification of suspected malware. Most solutions rely on function call patterns to determine whether a sample is exhibiting malicious behavior. These system and API calls are usually regarded individually and do not consider contextual information or process inter-dependencies. In addition, the patterns are often fixed in nature and do not adapt to changing circumstances on the system environment level. To address these shortcomings, this paper proposes a sentiment extraction and scoring system capable of learning the maliciousness inherent to n-grams of kernel events captured by a real-time monitoring agent. The approach is based on calculating the log likelihood ratio (LLR) of all identified n-grams, effectively determining neighboring sequences as well as assessing whether certain event combinations incline towards the benign or malicious. The extraction component automatically compiles a WordNet-like sentiment dictionary of events, which is subsequently used to score unknown traces of either individual processes, or a session in its entirety. The system was evaluated using a large set of real-world event traces collected on live corporate workstations as well as raw API call traces created in a dedicated malware analysis environment. While applicable to both scenarios, the introduced solution performed best for our abstracted kernel events, generating both new insight into malware–system interaction and assisting with the scoring of hitherto unknown application behavior.

Defining Malicious Behavior

In this paper we propose the use of formal methods to model malicious code behavior. The paradigm shift in malware detection from conventional, signature-based static methods to evaluating dynamic system behavior is motivated by the rising number and ever-increasing sophistication of malware currently in the wild. Because of advanced polymorphic and metamorphic techniques, a purely signature-based approach is no longer sufficient for accurate malware recognition. Automating the process of behavior analysis necessitates the use of formal methods. The modeling process is built upon two cornerstones: special system call execution traces generated through dynamic analysis of suspicious code and a self-defined taxonomy of (malicious) system activities. The formal model consists of two parts: A definition of malicious behavior in the form of combinations of tasks necessary to achieve a certain malign goal and of rules for translating each task into possible patterns of system calls. Both models are realized through formal grammars. The behavior model uses the tasks as the alphabet and the grammar rules define which patterns of activities can be used to accomplish certain high-level malicious goals. The translation model on the other hand contains an attributed context-free grammar for each task. The alphabet of each grammar consists of Windows system (API) calls, the grammar rules map each task to patterns of these calls. The attributes are used to convey information contained in the parameters of the individual calls.