Masashi Eto

nicter: An Incident Analysis System Toward Binding Network Monitoring with Malware Analysis

We have been developing the Network Incident analysis Center for Tactical Emergency Response (nicter), whose present focus is on detecting and identifying propagating malwares such as worms, viruses, and bots. The nicter presently monitors darknet, a set of unused IP addresses, to observe macroscopic trends of network threats. Meantime, it keeps capturing and analyzing malware executables in the wild for their microscopic analysis. Finally, these macroscopic and microscopic analysis results are correlated in order to identify the root cause of the detected network threats. This paper describes a brief overview of the nicter, and possible contributions to the worldwide observatory of malicious behavior and attack tools (WOMBAT).

An incident analysis system NICTER and its analysis engines based on data mining techniques

Malwares are spread all over cyberspace and often lead to serious security incidents. To grasp the present trends of malware activities, there are a number of ongoing network monitoring projects that collect large amount of data such as network traffic and IDS logs. These data need to be analyzed in depth since they potentially contain critical symptoms, such as an outbreak of new malware, a stealthy activity of botnet and a new type of attack on unknown vulnerability, etc. We have been developing the Network Incident analysis Center for Tactical Emergency Response (NICTER), which monitors a wide range of networks in real-time. The NICTER deploys several analysis engines taking advantage of data mining techniques in order to analyze the monitored traffics. This paper describes a brief overview of the NICTER, and its data mining based analysis engines, such as Change Point Detector (CPD), Self-Organizing Map analyzer (SOM analyzer) and Incident Forecast engine (IF).

DAEDALUS-VIZ: novel real-time 3D visualization for darknet monitoring-based alert system

A darknet is a set of unused IP addresses whose monitoring is an effective way of detecting malicious activities on the Internet. We have developed an alert system called DAEDALUS (direct alert environment for darknet and livenet unified security), which is based on large-scale darknet monitoring. This paper presents a novel real-time 3D visualization engine called DAEDALUS-VIZ that enables operators to grasp visually and in real time a complete overview of alert circumstances and provides highly flexible and tangible interactivity. We describe some case studies and evaluate the performance of DAEDALUS-VIZ.

nicter: a large-scale network incident analysis system: case studies for understanding threat landscape

We have been developing the Network Incident analysis Center for Tactical Emergency Response (nicter), whose objective is to detect and identify propagating malwares. The nicter mainly monitors darknet, a set of unused IP addresses, to observe global trends of network threats, while it captures and analyzes malware executables. By correlating the network threats with analysis results of malware, the nicter identifies the root causes (malwares) of the detected network threats. Through a long-term operation of the nicter for more than five years, we have achieved some key findings that would help us to understand the intentions of attackers and the comprehensive threat landscape of the Internet. With a focus on a well-knwon malware, i. e., W32.Downadup, this paper provides some practical case studies with considerations and consequently we could obtain a threat landscape that more than 60% of attacking hosts observed in our dark-net could be infected by W32.Downadup. As an evaluation, we confirmed that the result of the correlation analysis was correct in a rate of 86.18%.

Correlation Analysis between Spamming Botnets and Malware Infected Hosts

Many of recent cyber attacks are being launched by botnets for the purpose of carrying out large-scale cyber attacks such as spam emails, Distributed Denial of Service (DDoS), network scanning and so on. In many cases, these botnets consist of a lot of bots or zombie PCs which have been infected by a specific malware, and they try to propagate themselves into other victim systems through the Internet. In order to mitigate heavy damage of botnet based cyber attacks, it is needed to better understand the basic infrastructure of botnets as well as the underlying malwares of them. In this paper, we carried out correlation analysis between 10 spamming botnets identified by analyzing 3 weeks of spam emails in our previous work and malware infected hosts that observed at our darknets and honey pots. By comparing members (i.e., bots) of 10 spamming botnets with source hosts of dark net and honey pot traffic, we found that 7.2% ~ 37.5% of spamming botnets has been infected by four different malwares at least.

Design issues of an isolated sandbox used to analyze malwares

Recent viruses, worms, and bots, called malwares, often have anti-analysis functions such as mechanisms that confirm connectivity to certain Internet hosts and detect virtualized environments. We discuss how malwares can be kept alive in an analyzing environment by disabling their anti-analyzing mechanisms. To avoid any impacts to/from the Internet, we conclude that analyzing environments should be disconnected from the Internet but must be able to make malwares believe that they are connected to the real Internet. We also conclude that, for executing environments to analyze anti-virtualization malwares, they should not be virtualized but must be as easily reconstructable as a virtualized environment. To reconcile these cross-purposes, we propose an approach that consists of a mimetic Internet and a malware incubator with swappable actual nodes. We implemented a prototype system and conducted an experiment to test the adequacy of our approach.

A study on association rule mining of darknet big data

Global darknet monitoring provides an effective way to observe cyber-attacks that are significantly threatening network security and management. In this paper, we present a study on characterization of cyberattacks in the big stream data collected in a large scale distributed darknet using association rule learning. The experiment shows that association rule learning in the darknet stream data can support strategic cyberattack countermeasure in the following ways. First, statistics computed from malware-specific rules can lead to better understanding of the global trend of cyberattacks in the Internet. Second, strong association rules can lead to further insights into the nature of the attacking tools and hence expedite the diagnosis. Then, the discovery of emerging new attacks may lead to early detection and prompt prevention of pandemic incidents, preventing damage to the IT infrastructure and extensive financial loss. Finally, exploring the knowledge in the frequent attacking patterns can enable accurate prediction of future attacks from analyzed hosts, which could improve the performance of honeypot systems to collect more pertinent malware information using limited system and network resources.

DAEDALUS: Novel Application of Large-Scale Darknet Monitoring for Practical Protection of Live Networks

Large-scale darknet monitoring is an effective approach to grasp a global trend of malicious activities on the Internet, such as the world-wide spread of malwares. There, however, have been a gap between the darknet monitoring and actual security operations on live networks, namely the global trend has less direct contribution to protect the live networks. Therefore, we propose a novel application of large-scale darknet monitoring that significantly contributes to the security of live networks. In contrast to the conventional method, wherein the packets received from the outside are observed, we employ a large-scale distributed darknet that consists of several organizations that mutually observe the malicious packets transmitted from the inside of the organizations. Based on this approach, we have developed an alert system called DAEDALUS (direct alert environment for darknet and livenet unified security). We present the primary experimental results obtained from the actual deployment of DAEDALUS.

An Empirical Study of Spam : Analyzing Spam Sending Systems and Malicious Web Servers

Most recent spam emails are being sent by bots which often operate with others in the form of a botnet and in many cases, they contain URLs that navigate spam receivers to malicious Web servers for the purpose of carrying out various cyber attacks such as malware infection, phishing attacks, etc. In order to characterize the infrastructure of spam based attacks and identify botnets, previous research has been focused on clustering spam according to similarities based on email contents or URLs or their domain names. However, there is a fatal weakness in that the three criteria are easily influenced by changes in spam messages and trends. In this paper, we present a new spam clustering method based on IP addresses resolved from URLs within spam emails. By examining three weeks of spam gathered in our SMTP server, we observed that the accuracy of our clustering method is superior to that of domain name and URL based clustering methods, and we have obtained many useful results related to characteristics and clusters of spam that can be utilized for further analysis of spam based attacks.

A heuristic-based feature selection method for clustering spam emails

In recent years, in order to cope with spam based attacks, there have been many efforts made towards the clustering of spam emails. During the clustering process, many statistical features (e.g., the size of emails) are used for calculating similarities between spam emails. In many cases, however, some of the features may be redundant or contribute little to the clustering process. Feature selection is one of the most typical methods used to identify a subset of key features from an initial set. In this paper, we propose a heuristic-based feature selection method for clustering spam emails. Unlike the existing methods in that they make the combinations of given features and evaluate them using data mining and machine learning techniques, our method focuses on evaluating each feature according to only its value distribution in spam clusters. With our method, we identified 4 significant features which yielded a clustering accuracy of 86.33% with low time complexity.